Version: 0.16.16

AbstractDataContext

class great_expectations.data_context.AbstractDataContext(runtime_environment: Optional[dict] = None)#

Base class for all Data Contexts that contains shared functionality.

The class encapsulates most store / core components and convenience methods used to access them, meaning the majority of Data Context functionality lives here.

One of the primary responsibilities of the DataContext is managing CRUD operations for core GX objects:

Supported CRUD Methods#
	Stores	Datasources	ExpectationSuites	Checkpoints	Profilers
get	❌	✅	✅	✅	✅
add	✅	✅	✅	✅	✅
update	❌	✅	✅	✅	✅
add_or_update	❌	✅	✅	✅	✅
delete	✅	✅	✅	✅	✅

add_checkpoint(name: str = None, config_version: int | float = 1.0, template_name: str | None = None, module_name: str = 'great_expectations.checkpoint', class_name: str = 'Checkpoint', run_name_template: str | None = None, expectation_suite_name: str | None = None, batch_request: dict | None = None, action_list: Sequence[ActionDict] | None = None, evaluation_parameters: dict | None = None, runtime_configuration: dict | None = None, validations: list[dict] | None = None, profilers: list[dict] | None = None, site_names: str | list[str] | None = None, slack_webhook: str | None = None, notify_on: str | None = None, notify_with: str | list[str] | None = None, ge_cloud_id: str | None = None, expectation_suite_ge_cloud_id: str | None = None, default_validation_id: str | None = None, id: str | None = None, expectation_suite_id: str | None = None, validator: Validator | None = None, checkpoint: None = None) → great_expectations.checkpoint.checkpoint.Checkpoint#

add_checkpoint(name: None = None, config_version: int | float = 1.0, template_name: None = None, module_name: str = 'great_expectations.checkpoint', class_name: str = 'Checkpoint', run_name_template: None = None, expectation_suite_name: None = None, batch_request: None = None, action_list: Sequence[ActionDict] | None = None, evaluation_parameters: None = None, runtime_configuration: None = None, validations: None = None, profilers: None = None, site_names: None = None, slack_webhook: None = None, notify_on: None = None, notify_with: None = None, ge_cloud_id: None = None, expectation_suite_ge_cloud_id: None = None, default_validation_id: None = None, id: None = None, expectation_suite_id: None = None, validator: Validator | None = None, checkpoint: great_expectations.checkpoint.checkpoint.Checkpoint = None) → great_expectations.checkpoint.checkpoint.Checkpoint

Add a Checkpoint to the DataContext.

-Relevant Documentation Links -

Checkpoint

Parameters

name – The name to give the checkpoint.
config_version – The config version of this checkpoint.
template_name – The template to use in generating this checkpoint.
module_name – The module name to use in generating this checkpoint.
class_name – The class name to use in generating this checkpoint.
run_name_template – The run name template to use in generating this checkpoint.
expectation_suite_name – The expectation suite name to use in generating this checkpoint.
batch_request – The batch request to use in generating this checkpoint.
action_list – The action list to use in generating this checkpoint.
evaluation_parameters – The evaluation parameters to use in generating this checkpoint.
runtime_configuration – The runtime configuration to use in generating this checkpoint.
validations – The validations to use in generating this checkpoint.
profilers – The profilers to use in generating this checkpoint.
site_names – The site names to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.
slack_webhook – The slack webhook to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.
notify_on – The notify on setting to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.
notify_with – The notify with setting to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.
ge_cloud_id – The GE Cloud ID to use in generating this checkpoint.
expectation_suite_ge_cloud_id – The expectation suite GE Cloud ID to use in generating this checkpoint.
default_validation_id – The default validation ID to use in generating this checkpoint.
id –

The ID to use in generating this checkpoint (preferred over ge_cloud_id).

New in version 0.15.48: To be used in place of ge_cloud_id
expectation_suite_id –

The expectation suite ID to use in generating this checkpoint (preferred over expectation_suite_ge_cloud_id).

New in version 0.15.48: To be used in place of expectation_suite_ge_cloud_id
validator –

An existing validator used to generate a validations list.

New in version 0.16.15: Pass in an existing validator instead of individual validations
checkpoint –

An existing checkpoint you wish to persist.

New in version 0.15.48: Pass in an existing checkpoint instead of individual constructor args

Returns

The Checkpoint object created.

add_datasource(name: str = None, initialize: bool = True, save_changes: bool | None = None, datasource: None = None, **kwargs) → BaseDatasource | FluentDatasource | LegacyDatasource | None#

Add a new Datasource to the data context, with configuration provided as kwargs.

Relevant Documentation Links

Datasource

Parameters

name – the name of the new Datasource to add
initialize – if False, add the Datasource to the config, but do not initialize it, for example if a user needs to debug database connectivity.
save_changes –

should GX save the Datasource config?

Deprecated since version 0.15.32.
datasource –

an existing Datasource you wish to persist

New in version 0.15.49: Pass in an existing Datasource instead of individual constructor arguments
kwargs – the configuration for the new Datasource

Returns

Datasource instance added.

add_expectation_suite(expectation_suite_name: str, id: str | None = None, expectations: list[dict | ExpectationConfiguration] | None = None, evaluation_parameters: dict | None = None, data_asset_type: str | None = None, execution_engine_type: Type[ExecutionEngine] | None = None, meta: dict | None = None, expectation_suite: None = None) → great_expectations.core.expectation_suite.ExpectationSuite#

add_expectation_suite(expectation_suite_name: None = None, id: str | None = None, expectations: list[dict | ExpectationConfiguration] | None = None, evaluation_parameters: dict | None = None, data_asset_type: str | None = None, execution_engine_type: Type[ExecutionEngine] | None = None, meta: dict | None = None, expectation_suite: great_expectations.core.expectation_suite.ExpectationSuite = None) → great_expectations.core.expectation_suite.ExpectationSuite

Build a new ExpectationSuite and save it utilizing the context's underlying ExpectationsStore.

New in version 0.15.48.

Note that this method can be called by itself or run within the get_validator workflow.

When run with create_expectation_suite():

                                      expectation_suite_name = "genres_movies.fkey"context.create_expectation_suite(expectation_suite_name, overwrite_existing=True)batch = context.get_batch(expectation_suite_name=expectation_suite_name)
                                    

When run as part of get_validator():

                                      validator = context.get_validator(datasource_name="my_datasource",data_connector_name="whole_table",data_asset_name="my_table",create_expectation_suite_with_name="my_expectation_suite",)validator.expect_column_values_to_be_in_set("c1", [4,5,6])
                                    

Parameters

expectation_suite_name – The name of the suite to create.
id – Identifier to associate with this suite.
expectations – Expectation Configurations to associate with this suite.
evaluation_parameters – Evaluation parameters to be substituted when evaluating Expectations.
data_asset_type – Type of data asset to associate with this suite.
execution_engine_type – Name of the execution engine type.
meta – Metadata related to the suite.

Returns

A new ExpectationSuite built with provided input args.

Raises

DataContextError – A suite with the same name already exists (and overwrite_existing is not enabled).
ValueError – The arguments provided are invalid.

add_or_update_checkpoint(name: str = None, id: str | None = None, config_version: int | float = 1.0, template_name: str | None = None, module_name: str = 'great_expectations.checkpoint', class_name: str = 'Checkpoint', run_name_template: str | None = None, expectation_suite_name: str | None = None, batch_request: dict | None = None, action_list: Sequence[ActionDict] | None = None, evaluation_parameters: dict | None = None, runtime_configuration: dict | None = None, validations: list[dict] | None = None, profilers: list[dict] | None = None, site_names: str | list[str] | None = None, slack_webhook: str | None = None, notify_on: str | None = None, notify_with: str | list[str] | None = None, expectation_suite_id: str | None = None, default_validation_id: str | None = None, validator: Validator | None = None, checkpoint: None = None) → great_expectations.checkpoint.checkpoint.Checkpoint#

add_or_update_checkpoint(name: None = None, id: None = None, config_version: int | float = 1.0, template_name: None = None, module_name: str = 'great_expectations.checkpoint', class_name: str = 'Checkpoint', run_name_template: None = None, expectation_suite_name: None = None, batch_request: None = None, action_list: Sequence[ActionDict] | None = None, evaluation_parameters: None = None, runtime_configuration: None = None, validations: None = None, profilers: None = None, site_names: None = None, slack_webhook: None = None, notify_on: None = None, notify_with: None = None, expectation_suite_id: None = None, default_validation_id: None = None, validator: Validator | None = None, checkpoint: great_expectations.checkpoint.checkpoint.Checkpoint = None) → great_expectations.checkpoint.checkpoint.Checkpoint

Add a new Checkpoint or update an existing one on the context depending on whether it already exists or not.

New in version 0.15.48.

Parameters

name – The name to give the checkpoint.
id – The ID to associate with this checkpoint.
config_version – The config version of this checkpoint.
template_name – The template to use in generating this checkpoint.
module_name – The module name to use in generating this checkpoint.
class_name – The class name to use in generating this checkpoint.
run_name_template – The run name template to use in generating this checkpoint.
expectation_suite_name – The expectation suite name to use in generating this checkpoint.
batch_request – The batch request to use in generating this checkpoint.
action_list – The action list to use in generating this checkpoint.
evaluation_parameters – The evaluation parameters to use in generating this checkpoint.
runtime_configuration – The runtime configuration to use in generating this checkpoint.
validations – The validations to use in generating this checkpoint.
profilers – The profilers to use in generating this checkpoint.
site_names – The site names to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.
slack_webhook – The slack webhook to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.
notify_on – The notify on setting to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.
notify_with – The notify with setting to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.
expectation_suite_id – The expectation suite GE Cloud ID to use in generating this checkpoint.
default_validation_id – The default validation ID to use in generating this checkpoint.
validator –

An existing validator used to generate a validations list.

New in version 0.16.15: Pass in an existing validator instead of individual validations
checkpoint – An existing checkpoint you wish to persist.

Returns

A new Checkpoint or an updated once (depending on whether or not it existed before this method call).

add_or_update_datasource(name: str = None, datasource: None = None, **kwargs) → BaseDatasource | FluentDatasource | LegacyDatasource#

add_or_update_datasource(name: None = None, datasource: BaseDatasource | FluentDatasource | LegacyDatasource = None, **kwargs) → BaseDatasource | FluentDatasource | LegacyDatasource

Add a new Datasource or update an existing one on the context depending on whether

New in version 0.15.48.

it already exists or not. The configuration is provided as kwargs.

Parameters

name – The name of the Datasource to add or update.
datasource – an existing Datasource you wish to persist.
kwargs – Any relevant keyword args to use when adding or updating the target Datasource named name.

Returns

The Datasource added or updated by the input kwargs.

add_or_update_expectation_suite(expectation_suite_name: str, id: str | None = None, expectations: list[dict | ExpectationConfiguration] | None = None, evaluation_parameters: dict | None = None, data_asset_type: str | None = None, execution_engine_type: Type[ExecutionEngine] | None = None, meta: dict | None = None, expectation_suite: None = None) → great_expectations.core.expectation_suite.ExpectationSuite#

add_or_update_expectation_suite(expectation_suite_name: None = None, id: str | None = None, expectations: list[dict | ExpectationConfiguration] | None = None, evaluation_parameters: dict | None = None, data_asset_type: str | None = None, execution_engine_type: Type[ExecutionEngine] | None = None, meta: dict | None = None, expectation_suite: great_expectations.core.expectation_suite.ExpectationSuite = None) → great_expectations.core.expectation_suite.ExpectationSuite

Add a new ExpectationSuite or update an existing one on the context depending on whether it already exists or not.

New in version 0.15.48.

Parameters

expectation_suite_name – The name of the suite to create or replace.
id – Identifier to associate with this suite (ignored if updating existing suite).
expectations – Expectation Configurations to associate with this suite.
evaluation_parameters – Evaluation parameters to be substituted when evaluating Expectations.
data_asset_type – Type of Data Asset to associate with this suite.
execution_engine_type – Name of the Execution Engine type.
meta – Metadata related to the suite.
expectation_suite – The ExpectationSuite object you wish to persist.

Returns

The persisted ExpectationSuite.

add_or_update_profiler(name: str, config_version: float, rules: dict[str, dict], variables: dict | None = None, profiler: None = None) → great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler#

add_or_update_profiler(name: None = None, config_version: None = None, rules: None = None, variables: None = None, profiler: great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler = None) → great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler

Add a new Profiler or update an existing one on the context depending on whether it already exists or not.

New in version 0.15.48.

Parameters

name – The name of the RBP instance.
config_version – The version of the RBP (currently only 1.0 is supported).
rules – A set of dictionaries, each of which contains its own domain_builder, parameter_builders, and expectation_configuration_builders.
variables – Any variables to be substituted within the rules.
id – The id associated with the RBP instance (if applicable).
profiler – An existing RuleBasedProfiler to persist.

Returns

A new Profiler or an updated one (depending on whether or not it existed before this method call).

add_profiler(name: str, config_version: float, rules: dict[str, dict], variables: dict | None = None, profiler: None = None) → great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler#

add_profiler(name: None = None, config_version: None = None, rules: None = None, variables: None = None, profiler: great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler = None) → great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler

Constructs a Profiler, persists it utilizing the context's underlying ProfilerStore, and returns it to the user for subsequent usage.

Parameters

name – The name of the RBP instance.
config_version – The version of the RBP (currently only 1.0 is supported).
rules – A set of dictionaries, each of which contains its own domain_builder, parameter_builders, and expectation_configuration_builders.
variables – Any variables to be substituted within the rules.
profiler –

An existing RuleBasedProfiler to persist.

New in version 0.15.48: Pass in an existing profiler instead of individual constructor args

Returns

The persisted Profiler constructed by the input arguments.

add_store(store_name: str, store_config: StoreConfigTypedDict) → Store#

Add a new Store to the DataContext.

Parameters

store_name – the name to associate with the created store.
store_config – the config to use to construct the store.

Returns

The instantiated Store.

build_data_docs(site_names=None, resource_identifiers=None, dry_run=False, build_index: bool = True)#

Build Data Docs for your project.

Relevant Documentation Links

Data Docs

Parameters

site_names – if specified, build data docs only for these sites, otherwise, build all the sites specified in the context's config
resource_identifiers – a list of resource identifiers (ExpectationSuiteIdentifier, ValidationResultIdentifier). If specified, rebuild HTML (or other views the data docs sites are rendering) only for the resources in this list. This supports incremental build of data docs sites (e.g., when a new validation result is created) and avoids full rebuild.
dry_run – a flag, if True, the method returns a structure containing the URLs of the sites that would be built, but it does not build these sites.
build_index – a flag if False, skips building the index page

Returns

A dictionary with the names of the updated data documentation sites as keys and the location info of their index.html files as values

Raises

ClassInstantiationError – Site config in your Data Context config is not valid.

create_expectation_suite(expectation_suite_name: str, overwrite_existing: bool = False, **kwargs: Optional[dict]) → great_expectations.core.expectation_suite.ExpectationSuite#

Build a new ExpectationSuite and save it utilizing the context's underlying ExpectationsStore.

Deprecated since version 0.15.48: Part of the deprecated DataContext CRUD API

Note that this method can be called by itself or run within the get_validator workflow.

When run with create_expectation_suite():

expectation_suite_name = "version-0.16.16 genres_movies.fkey"
context.create_expectation_suite(expectation_suite_name, overwrite_existing=True)
batch = context.get_batch(
    expectation_suite_name=expectation_suite_name
)

                                        
                                      

When run as part of get_validator():

validator = context.get_validator(
    datasource_name="version-0.16.16 my_datasource",
    data_connector_name="version-0.16.16 whole_table",
    data_asset_name="version-0.16.16 my_table",
    create_expectation_suite_with_name="version-0.16.16 my_expectation_suite",
)
validator.expect_column_values_to_be_in_set("c1", [4,5,6])

                                        
                                      

Parameters

expectation_suite_name – The name of the suite to create.
overwrite_existing – Whether to overwrite if a suite with the given name already exists.
**kwargs – Any key-value arguments to pass to the store when persisting.

Returns

A new (empty) ExpectationSuite.

Raises

ValueError – The input overwrite_existing is of the wrong type.
DataContextError – A suite with the same name already exists (and overwrite_existing is not enabled).

delete_checkpoint(name: str | None = None, ge_cloud_id: str | None = None, id: str | None = None) → None#

Deletes a given Checkpoint by either name or id.

Parameters

name – The name of the target Checkpoint.
ge_cloud_id – The id associated with the target Checkpoint.
id –

The id associated with the target Checkpoint (preferred over ge_cloud_id).

New in version 0.15.48: To be used in place of ge_cloud_id

Raises

CheckpointNotFoundError – If the requested Checkpoint does not exist.

delete_datasource(datasource_name: Optional[str], save_changes: Optional[bool] = None) → None#

Delete a given Datasource by name.

Note that this method causes deletion from the underlying DatasourceStore. This can be overridden to only impact the Datasource cache through the deprecatedsave_changes argument.

Parameters

datasource_name – The name of the target datasource.
save_changes –

Should this change be persisted by the DatasourceStore?

Deprecated since version 0.15.32.

Raises

ValueError – The datasource_name isn't provided or cannot be found.

delete_expectation_suite(expectation_suite_name: str | None = None, ge_cloud_id: str | None = None, id: str | None = None) → None#

Delete specified expectation suite from data_context expectation store.

Parameters

expectation_suite_name – The name of the expectation suite to delete
ge_cloud_id – The identifier of the expectation suite to delete
id –

The identifier of the expectation suite to delete (preferred over ge_cloud_id)

New in version 0.15.48: To be used in place of ge_cloud_id

Returns

True for Success and False for Failure.

delete_profiler(name: str | None = None, ge_cloud_id: str | None = None, id: str | None = None) → None#

Deletes a given Profiler by either name or id.

Parameters

name – The name of the target Profiler.
ge_cloud_id – The id associated with the target Profiler.
id –

The id associated with the target Profiler (preferred over ge_cloud_id).

New in version 0.15.48: To be used in place of ge_cloud_id

Raises

ProfilerNotFoundError – If the requested Profiler does not exists.

delete_store(store_name: str) → None#

Delete an existing Store from the DataContext.

New in version 0.15.48.

Parameters: store_name – The name of the Store to be deleted.
Raises: StoreConfigurationError if the target Store is not found. –

Inspect datasource and batch kwargs generators to provide available data_asset objects.

Parameters

datasource_names – List of datasources for which to provide available data asset name objects. If None, return available data assets for all datasources.
batch_kwargs_generator_names – List of batch kwargs generators for which to provide available data_asset_name objects.

Returns

Dictionary describing available data assets

Return type

data_asset_names

Raises

ValueError – datasource_names is not None, a string, or list of strings.

get_batch(arg1: Optional[Any] = None, arg2: Optional[Any] = None, arg3: Optional[Any] = None, **kwargs) → Union[great_expectations.core.batch.Batch, great_expectations.data_asset.data_asset.DataAsset]#

Get exactly one batch, based on a variety of flexible input types.

The method get_batch is the main user-facing method for getting batches; it supports both the new (V3) and the Legacy (V2) Datasource schemas. The version-specific implementations are contained in "_get_batch_v2()" and "_get_batch_v3()", respectively, both of which are in the present module.

For the V3 API parameters, please refer to the signature and parameter description of method "_get_batch_v3()". For the Legacy usage, please refer to the signature and parameter description of the method "_get_batch_v2()".

Processing Steps:

Determine the version (possible values are "v3" or "v2").
Convert the positional arguments to the appropriate named arguments, based on the version.
Package the remaining arguments as variable keyword arguments (applies only to V3).
Call the version-specific method ("_get_batch_v3()" or "_get_batch_v2()") with the appropriate arguments.

Parameters

arg1 – the first positional argument (can take on various types)
arg2 – the second positional argument (can take on various types)
arg3 – the third positional argument (can take on various types)
**kwargs – variable arguments

Returns

Batch (V3) or DataAsset (V2) – the requested batch

get_batch_list(datasource_name: Optional[str] = None, data_connector_name: Optional[str] = None, data_asset_name: Optional[str] = None, batch_request: Optional[BatchRequestBase] = None, batch_data: Optional[Any] = None, data_connector_query: Optional[dict] = None, batch_identifiers: Optional[dict] = None, limit: Optional[int] = None, index: Optional[Union[int, list, tuple, slice, str]] = None, custom_filter_function: Optional[Callable] = None, sampling_method: Optional[str] = None, sampling_kwargs: Optional[dict] = None, splitter_method: Optional[str] = None, splitter_kwargs: Optional[dict] = None, runtime_parameters: Optional[dict] = None, query: Optional[str] = None, path: Optional[str] = None, batch_filter_parameters: Optional[dict] = None, batch_spec_passthrough: Optional[dict] = None, batch_request_options: Optional[Union[dict, BatchRequestOptions]] = None, **kwargs: Optional[dict]) → List[Batch]#

Get the list of zero or more batches, based on a variety of flexible input types.

get_batch_list is the main user-facing API for getting batches. In contrast to virtually all other methods in the class, it does not require typed or nested inputs. Instead, this method is intended to help the user pick the right parameters

This method attempts to return any number of batches, including an empty list.

Parameters

datasource_name – The name of the Datasource that defines the Data Asset to retrieve the batch for
data_connector_name – The Data Connector within the datasource for the Data Asset
data_asset_name – The name of the Data Asset within the Data Connector
batch_request – Encapsulates all the parameters used here to retrieve a BatchList. Use eitherbatch_request or the other params (but not both)
batch_data – Provides runtime data for the batch; is added as the key batch_data to the runtime_parameters dictionary of a BatchRequest
query – Provides runtime data for the batch; is added as the key query to the runtime_parameters dictionary of a BatchRequest
path – Provides runtime data for the batch; is added as the key path to the runtime_parameters dictionary of a BatchRequest
runtime_parameters – Specifies runtime parameters for the BatchRequest; can includes keys batch_data,query, and path
data_connector_query – Used to specify connector query parameters; specifically batch_filter_parameters,limit, index, and custom_filter_function
batch_identifiers – Any identifiers of batches for the BatchRequest
batch_filter_parameters – Filter parameters used in the data connector query
limit – Part of the data_connector_query, limits the number of batches in the batch list
index – Part of the data_connector_query, used to specify the index of which batch to return. Negative numbers retrieve from the end of the list (ex: -1 retrieves the last or latest batch)
custom_filter_function – A Callable function that accepts batch_identifiers and returns a bool
sampling_method – The method used to sample Batch data (see: Splitting and Sampling)
sampling_kwargs – Arguments for the sampling method
splitter_method – The method used to split the Data Asset into Batches
splitter_kwargs – Arguments for the splitting method
batch_spec_passthrough – Arguments specific to the ExecutionEngine that aid in Batch retrieval
batch_request_options – Options for FluentBatchRequest
**kwargs – Used to specify either batch_identifiers or batch_filter_parameters

Returns

(Batch) The list of requested Batch instances

Raises

DatasourceError – If the specified datasource_name does not exist in the DataContext
TypeError – If the specified types of the batch_request are not supported, or if thedatasource_name is not a str
ValueError – If more than one exclusive parameter is specified (ex: specifing more than one of batch_data, query or path)

get_checkpoint(name: str | None = None, ge_cloud_id: str | None = None, id: str | None = None) → Checkpoint#

Retrieves a given Checkpoint by either name or id.

Parameters

name – The name of the target Checkpoint.
ge_cloud_id – The id associated with the target Checkpoint.
id –

The id associated with the target Checkpoint (preferred over ge_cloud_id).

New in version 0.15.48: To be used in place of ge_cloud_id

Returns

The requested Checkpoint.

Raises

CheckpointNotFoundError – If the requested Checkpoint does not exist.

get_datasource(datasource_name: str = 'default') → BaseDatasource | FluentDatasource | LegacyDatasource#

Retrieve a given Datasource by name from the context's underlying DatasourceStore.

Parameters: datasource_name – The name of the target datasource.
Returns: The target datasource.
Raises: ValueError – The input datasource_name is None.

get_expectation_suite(expectation_suite_name: str | None = None, include_rendered_content: bool | None = None, ge_cloud_id: str | None = None) → ExpectationSuite#

Get an Expectation Suite by name.

Parameters

expectation_suite_name (str) – The name of the Expectation Suite
include_rendered_content (bool) – Whether to re-populate rendered_content for each ExpectationConfiguration.
ge_cloud_id (str) –

The GX Cloud ID for the Expectation Suite (unused)

Deprecated since version 0.15.45.

Returns

An existing ExpectationSuite

Raises

DataContextError – There is no expectation suite with the name provided

get_profiler(name: str | None = None, ge_cloud_id: str | None = None, id: str | None = None) → RuleBasedProfiler#

Retrieves a given Profiler by either name or id.

Parameters

name – The name of the target Profiler.
ge_cloud_id – The id associated with the target Profiler.
id –

The id associated with the target Profiler (preferred over ge_cloud_id).

New in version 0.15.48: To be used in place of ge_cloud_id

Returns

The requested Profiler.

Raises

ProfilerNotFoundError – If the requested Profiler does not exists.

get_validator(datasource_name: Optional[str] = None, data_connector_name: Optional[str] = None, data_asset_name: Optional[str] = None, batch: Optional[Batch] = None, batch_list: Optional[List[Batch]] = None, batch_request: Optional[Union[BatchRequestBase, FluentBatchRequest]] = None, batch_request_list: Optional[List[BatchRequestBase]] = None, batch_data: Optional[Any] = None, data_connector_query: Optional[Union[IDDict, dict]] = None, batch_identifiers: Optional[dict] = None, limit: Optional[int] = None, index: Optional[Union[int, list, tuple, slice, str]] = None, custom_filter_function: Optional[Callable] = None, sampling_method: Optional[str] = None, sampling_kwargs: Optional[dict] = None, splitter_method: Optional[str] = None, splitter_kwargs: Optional[dict] = None, runtime_parameters: Optional[dict] = None, query: Optional[str] = None, path: Optional[str] = None, batch_filter_parameters: Optional[dict] = None, expectation_suite_ge_cloud_id: Optional[str] = None, batch_spec_passthrough: Optional[dict] = None, expectation_suite_name: Optional[str] = None, expectation_suite: Optional[ExpectationSuite] = None, create_expectation_suite_with_name: Optional[str] = None, include_rendered_content: Optional[bool] = None, expectation_suite_id: Optional[str] = None, **kwargs) → Validator#

Retrieve a Validator with a batch list and an ExpectationSuite.

get_validator first calls get_batch_list to retrieve a batch list, then creates or retrieves an ExpectationSuite used to validate the Batches in the list.

Parameters

datasource_name – The name of the Datasource that defines the Data Asset to retrieve the batch for
data_connector_name – The Data Connector within the datasource for the Data Asset
data_asset_name – The name of the Data Asset within the Data Connector
batch – The Batch to use with the Validator
batch_list – The List of Batches to use with the Validator
batch_request – Encapsulates all the parameters used here to retrieve a BatchList. Use eitherbatch_request or the other params (but not both)
batch_request_list – A List of BatchRequest to use with the Validator
batch_data – Provides runtime data for the batch; is added as the key batch_data to the runtime_parameters dictionary of a BatchRequest
query – Provides runtime data for the batch; is added as the key query to the runtime_parameters dictionary of a BatchRequest
path – Provides runtime data for the batch; is added as the key path to the runtime_parameters dictionary of a BatchRequest
runtime_parameters – Specifies runtime parameters for the BatchRequest; can includes keys batch_data,query, and path
data_connector_query – Used to specify connector query parameters; specifically batch_filter_parameters,limit, index, and custom_filter_function
batch_identifiers – Any identifiers of batches for the BatchRequest
batch_filter_parameters – Filter parameters used in the data connector query
limit – Part of the data_connector_query, limits the number of batches in the batch list
index – Part of the data_connector_query, used to specify the index of which batch to return. Negative numbers retrieve from the end of the list (ex: -1 retrieves the last or latest batch)
custom_filter_function – A Callable function that accepts batch_identifiers and returns a bool
sampling_method – The method used to sample Batch data (see: Splitting and Sampling)
sampling_kwargs – Arguments for the sampling method
splitter_method – The method used to split the Data Asset into Batches
splitter_kwargs – Arguments for the splitting method
batch_spec_passthrough – Arguments specific to the ExecutionEngine that aid in Batch retrieval
expectation_suite_ge_cloud_id – The identifier of the ExpectationSuite to retrieve from the DataContext (can be used in place of expectation_suite_name)
expectation_suite_name – The name of the ExpectationSuite to retrieve from the DataContext
expectation_suite – The ExpectationSuite to use with the validator
create_expectation_suite_with_name – Creates a Validator with a new ExpectationSuite with the provided name
include_rendered_content – If True the ExpectationSuite will include rendered content when saved
**kwargs – Used to specify either batch_identifiers or batch_filter_parameters

Returns

A Validator with the specified Batch list and ExpectationSuite

Return type

Validator

Raises

DatasourceError – If the specified datasource_name does not exist in the DataContext
TypeError – If the specified types of the batch_request are not supported, or if thedatasource_name is not a str
ValueError – If more than one exclusive parameter is specified (ex: specifing more than one of batch_data, query or path), or if the ExpectationSuite cannot be created or retrieved using either the provided name or identifier

list_checkpoints() → Union[List[str], List[great_expectations.data_context.types.resource_identifiers.ConfigurationIdentifier]]#

List existing Checkpoint identifiers on this context.

Returns: Either a list of strings or ConfigurationIdentifiers depending on the environment and context type.

list_datasources() → List[dict]#

List the configurations of the datasources associated with this context.

Note that any sensitive values are obfuscated before being returned.

Returns: A list of dictionaries representing datasource configurations.

list_expectation_suite_names() → List[str]#

Lists the available expectation suite names.

Returns: A list of suite names (sorted in alphabetic order).

Validate using an existing Checkpoint.

Parameters

checkpoint_name – The name of a Checkpoint defined via the CLI or by manually creating a yml file
template_name – The name of a Checkpoint template to retrieve from the CheckpointStore
run_name_template – The template to use for run_name
expectation_suite_name – Expectation suite to be used by Checkpoint run
batch_request – Batch request to be used by Checkpoint run
action_list – List of actions to be performed by the Checkpoint
evaluation_parameters – $parameter_name syntax references to be evaluated at runtime
runtime_configuration – Runtime configuration override parameters
validations – Validations to be performed by the Checkpoint run
profilers – Profilers to be used by the Checkpoint run
run_id – The run_id for the validation; if None, a default value will be used
run_name – The run_name for the validation; if None, a default value will be used
run_time – The date/time of the run
result_format – One of several supported formatting directives for expectation validation results
ge_cloud_id – Great Expectations Cloud id for the checkpoint
expectation_suite_ge_cloud_id – Great Expectations Cloud id for the expectation suite
id –

Great Expectations Cloud id for the checkpoint (preferred over ge_cloud_id)

New in version 0.15.48: To be used in place of ge_cloud_id
expectation_suite_id –

Great Expectations Cloud id for the expectation suite (preferred over expectation_suite_ge_cloud_id)

New in version 0.15.48: To be used in place of expectation_suite_ge_cloud_id
**kwargs – Additional kwargs to pass to the validation operator

Returns

CheckpointResult

save_expectation_suite(expectation_suite: great_expectations.core.expectation_suite.ExpectationSuite, expectation_suite_name: Optional[str] = None, overwrite_existing: bool = True, include_rendered_content: Optional[bool] = None, **kwargs: Optional[dict]) → None#

Save the provided ExpectationSuite into the DataContext using the configured ExpectationStore.

Deprecated since version 0.15.48: Part of the deprecated DataContext CRUD API

Parameters

expectation_suite – The ExpectationSuite to save.
expectation_suite_name – The name of this ExpectationSuite. If no name is provided, the name will be read from the suite.
overwrite_existing – Whether to overwrite the suite if it already exists.
include_rendered_content – Whether to save the prescriptive rendered content for each expectation.
kwargs – Additional parameters, unused

Returns

None

Raises

DataContextError – If a suite with the same name exists and overwrite_existing is set to False.

test_yaml_config(yaml_config: str, name: Optional[str] = None, class_name: Optional[str] = None, runtime_environment: Optional[dict] = None, pretty_print: bool = True, return_mode: Literal['instantiated_class', 'report_object'] = 'instantiated_class', shorten_tracebacks: bool = False)#

Convenience method for testing yaml configs.

test_yaml_config is a convenience method for configuring the moving parts of a Great Expectations deployment. It allows you to quickly test out configs for system components, especially Datasources, Checkpoints, and Stores.

For many deployments of Great Expectations, these components (plus Expectations) are the only ones you'll need.

test_yaml_config is mainly intended for use within notebooks and tests.

Relevant Documentation Links

Parameters

yaml_config – A string containing the yaml config to be tested
name – Optional name of the component to instantiate
class_name – Optional, overridden if provided in the config
runtime_environment – Optional override for config items
pretty_print – Determines whether to print human-readable output
return_mode – Determines what type of object test_yaml_config will return. Valid modes are "instantiated_class" and "report_object"
shorten_tracebacks – If true, catch any errors during instantiation and print only the last element of the traceback stack. This can be helpful for rapid iteration on configs in a notebook, because it can remove the need to scroll up and down a lot.

Returns

The instantiated component (e.g. a Datasource) OR a json object containing metadata from the component's self_check method. The returned object is determined by return_mode.

update_checkpoint(checkpoint: great_expectations.checkpoint.checkpoint.Checkpoint) → great_expectations.checkpoint.checkpoint.Checkpoint#

Update a Checkpoint that already exists.

New in version 0.15.48.

Parameters: checkpoint – The checkpoint to use to update.
Raises: DataContextError – A suite with the given name does not already exist.
Returns: The updated Checkpoint.

Updates a Datasource that already exists in the store.

Parameters

datasource – The Datasource object to update.
save_changes – do I save changes to disk?

Returns

The updated Datasource.

update_expectation_suite(expectation_suite: great_expectations.core.expectation_suite.ExpectationSuite) → great_expectations.core.expectation_suite.ExpectationSuite#

Update an ExpectationSuite that already exists.

New in version 0.15.48.

Parameters: expectation_suite – The suite to use to update.
Raises: DataContextError – A suite with the given name does not already exist.

update_profiler(profiler: great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler) → great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler#

Update a Profiler that already exists.

New in version 0.15.48.

Parameters: profiler – The profiler to use to update.
Raises: ProfilerNotFoundError – A profiler with the given name/id does not already exist.

update_project_config(project_config: DataContextConfig | Mapping) → DataContextConfig#

Update the context's config with the values from another config object.

Parameters: project_config – The config to use to update the context's internal state.
Returns: The updated project config.

view_validation_result(result: CheckpointResult) → None#

New in version 0.16.15.

Opens a validation result in a browser.

Parameters: result – The result of a Checkpoint run.