Skip to main content
Version: 0.16.16

AbstractDataContext

class great_expectations.data_context.AbstractDataContext(runtime_environment: Optional[dict] = None)#

Base class for all Data Contexts that contains shared functionality.

The class encapsulates most store / core components and convenience methods used to access them, meaning the majority of Data Context functionality lives here.

One of the primary responsibilities of the DataContext is managing CRUD operations for core GX objects:

Supported CRUD Methods#

Stores

Datasources

ExpectationSuites

Checkpoints

Profilers

get

add

update

add_or_update

delete

add_checkpoint(name: str = None, config_version: int | float = 1.0, template_name: str | None = None, module_name: str = 'great_expectations.checkpoint', class_name: str = 'Checkpoint', run_name_template: str | None = None, expectation_suite_name: str | None = None, batch_request: dict | None = None, action_list: Sequence[ActionDict] | None = None, evaluation_parameters: dict | None = None, runtime_configuration: dict | None = None, validations: list[dict] | None = None, profilers: list[dict] | None = None, site_names: str | list[str] | None = None, slack_webhook: str | None = None, notify_on: str | None = None, notify_with: str | list[str] | None = None, ge_cloud_id: str | None = None, expectation_suite_ge_cloud_id: str | None = None, default_validation_id: str | None = None, id: str | None = None, expectation_suite_id: str | None = None, validator: Validator | None = None, checkpoint: None = None) great_expectations.checkpoint.checkpoint.Checkpoint#
add_checkpoint(name: None = None, config_version: int | float = 1.0, template_name: None = None, module_name: str = 'great_expectations.checkpoint', class_name: str = 'Checkpoint', run_name_template: None = None, expectation_suite_name: None = None, batch_request: None = None, action_list: Sequence[ActionDict] | None = None, evaluation_parameters: None = None, runtime_configuration: None = None, validations: None = None, profilers: None = None, site_names: None = None, slack_webhook: None = None, notify_on: None = None, notify_with: None = None, ge_cloud_id: None = None, expectation_suite_ge_cloud_id: None = None, default_validation_id: None = None, id: None = None, expectation_suite_id: None = None, validator: Validator | None = None, checkpoint: great_expectations.checkpoint.checkpoint.Checkpoint = None) great_expectations.checkpoint.checkpoint.Checkpoint

Add a Checkpoint to the DataContext.

-Relevant Documentation Links -
Parameters
  • name – The name to give the checkpoint.

  • config_version – The config version of this checkpoint.

  • template_name – The template to use in generating this checkpoint.

  • module_name – The module name to use in generating this checkpoint.

  • class_name – The class name to use in generating this checkpoint.

  • run_name_template – The run name template to use in generating this checkpoint.

  • expectation_suite_name – The expectation suite name to use in generating this checkpoint.

  • batch_request – The batch request to use in generating this checkpoint.

  • action_list – The action list to use in generating this checkpoint.

  • evaluation_parameters – The evaluation parameters to use in generating this checkpoint.

  • runtime_configuration – The runtime configuration to use in generating this checkpoint.

  • validations – The validations to use in generating this checkpoint.

  • profilers – The profilers to use in generating this checkpoint.

  • site_names – The site names to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.

  • slack_webhook – The slack webhook to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.

  • notify_on – The notify on setting to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.

  • notify_with – The notify with setting to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.

  • ge_cloud_id – The GE Cloud ID to use in generating this checkpoint.

  • expectation_suite_ge_cloud_id – The expectation suite GE Cloud ID to use in generating this checkpoint.

  • default_validation_id – The default validation ID to use in generating this checkpoint.

  • id

    The ID to use in generating this checkpoint (preferred over ge_cloud_id).

    New in version 0.15.48: To be used in place of ge_cloud_id

  • expectation_suite_id

    The expectation suite ID to use in generating this checkpoint (preferred over expectation_suite_ge_cloud_id).

    New in version 0.15.48: To be used in place of expectation_suite_ge_cloud_id

  • validator

    An existing validator used to generate a validations list.

    New in version 0.16.15: Pass in an existing validator instead of individual validations

  • checkpoint

    An existing checkpoint you wish to persist.

    New in version 0.15.48: Pass in an existing checkpoint instead of individual constructor args

Returns

The Checkpoint object created.

add_datasource(name: str = None, initialize: bool = True, save_changes: bool | None = None, datasource: None = None, **kwargs) BaseDatasource | FluentDatasource | LegacyDatasource | None#
add_datasource(name: None = None, initialize: bool = True, save_changes: bool | None = None, datasource: BaseDatasource | FluentDatasource | LegacyDatasource = None, **kwargs) BaseDatasource | FluentDatasource | LegacyDatasource | None

Add a new Datasource to the data context, with configuration provided as kwargs.

Relevant Documentation Links
Parameters
  • name – the name of the new Datasource to add

  • initialize – if False, add the Datasource to the config, but do not initialize it, for example if a user needs to debug database connectivity.

  • save_changes

    should GX save the Datasource config?

    Deprecated since version 0.15.32.

  • datasource

    an existing Datasource you wish to persist

    New in version 0.15.49: Pass in an existing Datasource instead of individual constructor arguments

  • kwargs – the configuration for the new Datasource

Returns

Datasource instance added.

add_expectation_suite(expectation_suite_name: str, id: str | None = None, expectations: list[dict | ExpectationConfiguration] | None = None, evaluation_parameters: dict | None = None, data_asset_type: str | None = None, execution_engine_type: Type[ExecutionEngine] | None = None, meta: dict | None = None, expectation_suite: None = None) great_expectations.core.expectation_suite.ExpectationSuite#
add_expectation_suite(expectation_suite_name: None = None, id: str | None = None, expectations: list[dict | ExpectationConfiguration] | None = None, evaluation_parameters: dict | None = None, data_asset_type: str | None = None, execution_engine_type: Type[ExecutionEngine] | None = None, meta: dict | None = None, expectation_suite: great_expectations.core.expectation_suite.ExpectationSuite = None) great_expectations.core.expectation_suite.ExpectationSuite

Build a new ExpectationSuite and save it utilizing the context's underlying ExpectationsStore.

New in version 0.15.48.

Note that this method can be called by itself or run within the get_validator workflow.

When run with create_expectation_suite():

expectation_suite_name = "genres_movies.fkey"context.create_expectation_suite(expectation_suite_name, overwrite_existing=True)batch = context.get_batch(expectation_suite_name=expectation_suite_name)

When run as part of get_validator():

validator = context.get_validator(datasource_name="my_datasource",data_connector_name="whole_table",data_asset_name="my_table",create_expectation_suite_with_name="my_expectation_suite",)validator.expect_column_values_to_be_in_set("c1", [4,5,6])
Parameters
  • expectation_suite_name – The name of the suite to create.

  • id – Identifier to associate with this suite.

  • expectations – Expectation Configurations to associate with this suite.

  • evaluation_parameters – Evaluation parameters to be substituted when evaluating Expectations.

  • data_asset_type – Type of data asset to associate with this suite.

  • execution_engine_type – Name of the execution engine type.

  • meta – Metadata related to the suite.

Returns

A new ExpectationSuite built with provided input args.

Raises
  • DataContextError – A suite with the same name already exists (and overwrite_existing is not enabled).

  • ValueError – The arguments provided are invalid.

add_or_update_checkpoint(name: str = None, id: str | None = None, config_version: int | float = 1.0, template_name: str | None = None, module_name: str = 'great_expectations.checkpoint', class_name: str = 'Checkpoint', run_name_template: str | None = None, expectation_suite_name: str | None = None, batch_request: dict | None = None, action_list: Sequence[ActionDict] | None = None, evaluation_parameters: dict | None = None, runtime_configuration: dict | None = None, validations: list[dict] | None = None, profilers: list[dict] | None = None, site_names: str | list[str] | None = None, slack_webhook: str | None = None, notify_on: str | None = None, notify_with: str | list[str] | None = None, expectation_suite_id: str | None = None, default_validation_id: str | None = None, validator: Validator | None = None, checkpoint: None = None) great_expectations.checkpoint.checkpoint.Checkpoint#
add_or_update_checkpoint(name: None = None, id: None = None, config_version: int | float = 1.0, template_name: None = None, module_name: str = 'great_expectations.checkpoint', class_name: str = 'Checkpoint', run_name_template: None = None, expectation_suite_name: None = None, batch_request: None = None, action_list: Sequence[ActionDict] | None = None, evaluation_parameters: None = None, runtime_configuration: None = None, validations: None = None, profilers: None = None, site_names: None = None, slack_webhook: None = None, notify_on: None = None, notify_with: None = None, expectation_suite_id: None = None, default_validation_id: None = None, validator: Validator | None = None, checkpoint: great_expectations.checkpoint.checkpoint.Checkpoint = None) great_expectations.checkpoint.checkpoint.Checkpoint

Add a new Checkpoint or update an existing one on the context depending on whether it already exists or not.

New in version 0.15.48.

Parameters
  • name – The name to give the checkpoint.

  • id – The ID to associate with this checkpoint.

  • config_version – The config version of this checkpoint.

  • template_name – The template to use in generating this checkpoint.

  • module_name – The module name to use in generating this checkpoint.

  • class_name – The class name to use in generating this checkpoint.

  • run_name_template – The run name template to use in generating this checkpoint.

  • expectation_suite_name – The expectation suite name to use in generating this checkpoint.

  • batch_request – The batch request to use in generating this checkpoint.

  • action_list – The action list to use in generating this checkpoint.

  • evaluation_parameters – The evaluation parameters to use in generating this checkpoint.

  • runtime_configuration – The runtime configuration to use in generating this checkpoint.

  • validations – The validations to use in generating this checkpoint.

  • profilers – The profilers to use in generating this checkpoint.

  • site_names – The site names to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.

  • slack_webhook – The slack webhook to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.

  • notify_on – The notify on setting to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.

  • notify_with – The notify with setting to use in generating this checkpoint. This is only used for SimpleCheckpoint configuration.

  • expectation_suite_id – The expectation suite GE Cloud ID to use in generating this checkpoint.

  • default_validation_id – The default validation ID to use in generating this checkpoint.

  • validator

    An existing validator used to generate a validations list.

    New in version 0.16.15: Pass in an existing validator instead of individual validations

  • checkpoint – An existing checkpoint you wish to persist.

Returns

A new Checkpoint or an updated once (depending on whether or not it existed before this method call).

add_or_update_datasource(name: str = None, datasource: None = None, **kwargs) BaseDatasource | FluentDatasource | LegacyDatasource#
add_or_update_datasource(name: None = None, datasource: BaseDatasource | FluentDatasource | LegacyDatasource = None, **kwargs) BaseDatasource | FluentDatasource | LegacyDatasource

Add a new Datasource or update an existing one on the context depending on whether

New in version 0.15.48.

it already exists or not. The configuration is provided as kwargs.

Parameters
  • name – The name of the Datasource to add or update.

  • datasource – an existing Datasource you wish to persist.

  • kwargs – Any relevant keyword args to use when adding or updating the target Datasource named name.

Returns

The Datasource added or updated by the input kwargs.

add_or_update_expectation_suite(expectation_suite_name: str, id: str | None = None, expectations: list[dict | ExpectationConfiguration] | None = None, evaluation_parameters: dict | None = None, data_asset_type: str | None = None, execution_engine_type: Type[ExecutionEngine] | None = None, meta: dict | None = None, expectation_suite: None = None) great_expectations.core.expectation_suite.ExpectationSuite#
add_or_update_expectation_suite(expectation_suite_name: None = None, id: str | None = None, expectations: list[dict | ExpectationConfiguration] | None = None, evaluation_parameters: dict | None = None, data_asset_type: str | None = None, execution_engine_type: Type[ExecutionEngine] | None = None, meta: dict | None = None, expectation_suite: great_expectations.core.expectation_suite.ExpectationSuite = None) great_expectations.core.expectation_suite.ExpectationSuite

Add a new ExpectationSuite or update an existing one on the context depending on whether it already exists or not.

New in version 0.15.48.

Parameters
  • expectation_suite_name – The name of the suite to create or replace.

  • id – Identifier to associate with this suite (ignored if updating existing suite).

  • expectations – Expectation Configurations to associate with this suite.

  • evaluation_parameters – Evaluation parameters to be substituted when evaluating Expectations.

  • data_asset_type – Type of Data Asset to associate with this suite.

  • execution_engine_type – Name of the Execution Engine type.

  • meta – Metadata related to the suite.

  • expectation_suite – The ExpectationSuite object you wish to persist.

Returns

The persisted ExpectationSuite.

add_or_update_profiler(name: str, config_version: float, rules: dict[str, dict], variables: dict | None = None, profiler: None = None) great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler#
add_or_update_profiler(name: None = None, config_version: None = None, rules: None = None, variables: None = None, profiler: great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler = None) great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler

Add a new Profiler or update an existing one on the context depending on whether it already exists or not.

New in version 0.15.48.

Parameters
  • name – The name of the RBP instance.

  • config_version – The version of the RBP (currently only 1.0 is supported).

  • rules – A set of dictionaries, each of which contains its own domain_builder, parameter_builders, and expectation_configuration_builders.

  • variables – Any variables to be substituted within the rules.

  • id – The id associated with the RBP instance (if applicable).

  • profiler – An existing RuleBasedProfiler to persist.

Returns

A new Profiler or an updated one (depending on whether or not it existed before this method call).

add_profiler(name: str, config_version: float, rules: dict[str, dict], variables: dict | None = None, profiler: None = None) great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler#
add_profiler(name: None = None, config_version: None = None, rules: None = None, variables: None = None, profiler: great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler = None) great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler

Constructs a Profiler, persists it utilizing the context's underlying ProfilerStore, and returns it to the user for subsequent usage.

Parameters
  • name – The name of the RBP instance.

  • config_version – The version of the RBP (currently only 1.0 is supported).

  • rules – A set of dictionaries, each of which contains its own domain_builder, parameter_builders, and expectation_configuration_builders.

  • variables – Any variables to be substituted within the rules.

  • profiler

    An existing RuleBasedProfiler to persist.

    New in version 0.15.48: Pass in an existing profiler instead of individual constructor args

Returns

The persisted Profiler constructed by the input arguments.

add_store(store_name: str, store_config: StoreConfigTypedDict) Store#

Add a new Store to the DataContext.

Parameters
  • store_name – the name to associate with the created store.

  • store_config – the config to use to construct the store.

Returns

The instantiated Store.

build_data_docs(site_names=None, resource_identifiers=None, dry_run=False, build_index: bool = True)#

Build Data Docs for your project.

Relevant Documentation Links
Parameters
  • site_names – if specified, build data docs only for these sites, otherwise, build all the sites specified in the context's config

  • resource_identifiers – a list of resource identifiers (ExpectationSuiteIdentifier, ValidationResultIdentifier). If specified, rebuild HTML (or other views the data docs sites are rendering) only for the resources in this list. This supports incremental build of data docs sites (e.g., when a new validation result is created) and avoids full rebuild.

  • dry_run – a flag, if True, the method returns a structure containing the URLs of the sites that would be built, but it does not build these sites.

  • build_index – a flag if False, skips building the index page

Returns

A dictionary with the names of the updated data documentation sites as keys and the location info of their index.html files as values

Raises

ClassInstantiationError – Site config in your Data Context config is not valid.

create_expectation_suite(expectation_suite_name: str, overwrite_existing: bool = False, **kwargs: Optional[dict]) great_expectations.core.expectation_suite.ExpectationSuite#

Build a new ExpectationSuite and save it utilizing the context's underlying ExpectationsStore.

Deprecated since version 0.15.48: Part of the deprecated DataContext CRUD API

Note that this method can be called by itself or run within the get_validator workflow.

When run with create_expectation_suite():

expectation_suite_name = "version-0.16.16 genres_movies.fkey"
context.create_expectation_suite(expectation_suite_name, overwrite_existing=True)
batch = context.get_batch(
expectation_suite_name=expectation_suite_name
)

When run as part of get_validator():

validator = context.get_validator(
datasource_name="version-0.16.16 my_datasource",
data_connector_name="version-0.16.16 whole_table",
data_asset_name="version-0.16.16 my_table",
create_expectation_suite_with_name="version-0.16.16 my_expectation_suite",
)
validator.expect_column_values_to_be_in_set("c1", [4,5,6])

Parameters
  • expectation_suite_name – The name of the suite to create.

  • overwrite_existing – Whether to overwrite if a suite with the given name already exists.

  • **kwargs – Any key-value arguments to pass to the store when persisting.

Returns

A new (empty) ExpectationSuite.

Raises
  • ValueError – The input overwrite_existing is of the wrong type.

  • DataContextError – A suite with the same name already exists (and overwrite_existing is not enabled).

delete_checkpoint(name: str | None = None, ge_cloud_id: str | None = None, id: str | None = None) None#

Deletes a given Checkpoint by either name or id.

Parameters
  • name – The name of the target Checkpoint.

  • ge_cloud_id – The id associated with the target Checkpoint.

  • id

    The id associated with the target Checkpoint (preferred over ge_cloud_id).

    New in version 0.15.48: To be used in place of ge_cloud_id

Raises

CheckpointNotFoundError – If the requested Checkpoint does not exist.

delete_datasource(datasource_name: Optional[str], save_changes: Optional[bool] = None) None#

Delete a given Datasource by name.

Note that this method causes deletion from the underlying DatasourceStore. This can be overridden to only impact the Datasource cache through the deprecatedsave_changes argument.

Parameters
  • datasource_name – The name of the target datasource.

  • save_changes

    Should this change be persisted by the DatasourceStore?

    Deprecated since version 0.15.32.

Raises

ValueError – The datasource_name isn't provided or cannot be found.

delete_expectation_suite(expectation_suite_name: str | None = None, ge_cloud_id: str | None = None, id: str | None = None) None#

Delete specified expectation suite from data_context expectation store.

Parameters
  • expectation_suite_name – The name of the expectation suite to delete

  • ge_cloud_id – The identifier of the expectation suite to delete

  • id

    The identifier of the expectation suite to delete (preferred over ge_cloud_id)

    New in version 0.15.48: To be used in place of ge_cloud_id

Returns

True for Success and False for Failure.

delete_profiler(name: str | None = None, ge_cloud_id: str | None = None, id: str | None = None) None#

Deletes a given Profiler by either name or id.

Parameters
  • name – The name of the target Profiler.

  • ge_cloud_id – The id associated with the target Profiler.

  • id

    The id associated with the target Profiler (preferred over ge_cloud_id).

    New in version 0.15.48: To be used in place of ge_cloud_id

Raises

ProfilerNotFoundError – If the requested Profiler does not exists.

delete_store(store_name: str) None#

Delete an existing Store from the DataContext.

New in version 0.15.48.

Parameters

store_name – The name of the Store to be deleted.

Raises

StoreConfigurationError if the target Store is not found.

get_available_data_asset_names(datasource_names: str | list[str] | None = None, batch_kwargs_generator_names: str | list[str] | None = None) dict[str, BlockConfigDataAssetNames | FluentDataAssetNames]#

Inspect datasource and batch kwargs generators to provide available data_asset objects.

Parameters
  • datasource_names – List of datasources for which to provide available data asset name objects. If None, return available data assets for all datasources.

  • batch_kwargs_generator_names – List of batch kwargs generators for which to provide available data_asset_name objects.

Returns

Dictionary describing available data assets

Return type

data_asset_names

Raises

ValueErrordatasource_names is not None, a string, or list of strings.

get_batch(arg1: Optional[Any] = None, arg2: Optional[Any] = None, arg3: Optional[Any] = None, **kwargs) Union[great_expectations.core.batch.Batch, great_expectations.data_asset.data_asset.DataAsset]#

Get exactly one batch, based on a variety of flexible input types.

The method get_batch is the main user-facing method for getting batches; it supports both the new (V3) and the Legacy (V2) Datasource schemas. The version-specific implementations are contained in "_get_batch_v2()" and "_get_batch_v3()", respectively, both of which are in the present module.

For the V3 API parameters, please refer to the signature and parameter description of method "_get_batch_v3()". For the Legacy usage, please refer to the signature and parameter description of the method "_get_batch_v2()".

Processing Steps:
  1. Determine the version (possible values are "v3" or "v2").

  2. Convert the positional arguments to the appropriate named arguments, based on the version.

  3. Package the remaining arguments as variable keyword arguments (applies only to V3).

  4. Call the version-specific method ("_get_batch_v3()" or "_get_batch_v2()") with the appropriate arguments.

Parameters
  • arg1 – the first positional argument (can take on various types)

  • arg2 – the second positional argument (can take on various types)

  • arg3 – the third positional argument (can take on various types)

  • **kwargs – variable arguments

Returns

Batch (V3) or DataAsset (V2) – the requested batch

get_batch_list(datasource_name: Optional[str] = None, data_connector_name: Optional[str] = None, data_asset_name: Optional[str] = None, batch_request: Optional[BatchRequestBase] = None, batch_data: Optional[Any] = None, data_connector_query: Optional[dict] = None, batch_identifiers: Optional[dict] = None, limit: Optional[int] = None, index: Optional[Union[int, list, tuple, slice, str]] = None, custom_filter_function: Optional[Callable] = None, sampling_method: Optional[str] = None, sampling_kwargs: Optional[dict] = None, splitter_method: Optional[str] = None, splitter_kwargs: Optional[dict] = None, runtime_parameters: Optional[dict] = None, query: Optional[str] = None, path: Optional[str] = None, batch_filter_parameters: Optional[dict] = None, batch_spec_passthrough: Optional[dict] = None, batch_request_options: Optional[Union[dict, BatchRequestOptions]] = None, **kwargs: Optional[dict]) List[Batch]#

Get the list of zero or more batches, based on a variety of flexible input types.

get_batch_list is the main user-facing API for getting batches. In contrast to virtually all other methods in the class, it does not require typed or nested inputs. Instead, this method is intended to help the user pick the right parameters

This method attempts to return any number of batches, including an empty list.

Parameters
  • datasource_name – The name of the Datasource that defines the Data Asset to retrieve the batch for

  • data_connector_name – The Data Connector within the datasource for the Data Asset

  • data_asset_name – The name of the Data Asset within the Data Connector

  • batch_request – Encapsulates all the parameters used here to retrieve a BatchList. Use eitherbatch_request or the other params (but not both)

  • batch_data – Provides runtime data for the batch; is added as the key batch_data to the runtime_parameters dictionary of a BatchRequest

  • query – Provides runtime data for the batch; is added as the key query to the runtime_parameters dictionary of a BatchRequest

  • path – Provides runtime data for the batch; is added as the key path to the runtime_parameters dictionary of a BatchRequest

  • runtime_parameters – Specifies runtime parameters for the BatchRequest; can includes keys batch_data,query, and path

  • data_connector_query – Used to specify connector query parameters; specifically batch_filter_parameters,limit, index, and custom_filter_function

  • batch_identifiers – Any identifiers of batches for the BatchRequest

  • batch_filter_parameters – Filter parameters used in the data connector query

  • limit – Part of the data_connector_query, limits the number of batches in the batch list

  • index – Part of the data_connector_query, used to specify the index of which batch to return. Negative numbers retrieve from the end of the list (ex: -1 retrieves the last or latest batch)

  • custom_filter_function – A Callable function that accepts batch_identifiers and returns a bool

  • sampling_method – The method used to sample Batch data (see: Splitting and Sampling)

  • sampling_kwargs – Arguments for the sampling method

  • splitter_method – The method used to split the Data Asset into Batches

  • splitter_kwargs – Arguments for the splitting method

  • batch_spec_passthrough – Arguments specific to the ExecutionEngine that aid in Batch retrieval

  • batch_request_options – Options for FluentBatchRequest

  • **kwargs – Used to specify either batch_identifiers or batch_filter_parameters

Returns

(Batch) The list of requested Batch instances

Raises
  • DatasourceError – If the specified datasource_name does not exist in the DataContext

  • TypeError – If the specified types of the batch_request are not supported, or if thedatasource_name is not a str

  • ValueError – If more than one exclusive parameter is specified (ex: specifing more than one of batch_data, query or path)

get_checkpoint(name: str | None = None, ge_cloud_id: str | None = None, id: str | None = None) Checkpoint#

Retrieves a given Checkpoint by either name or id.

Parameters
  • name – The name of the target Checkpoint.

  • ge_cloud_id – The id associated with the target Checkpoint.

  • id

    The id associated with the target Checkpoint (preferred over ge_cloud_id).

    New in version 0.15.48: To be used in place of ge_cloud_id

Returns

The requested Checkpoint.

Raises

CheckpointNotFoundError – If the requested Checkpoint does not exist.

get_datasource(datasource_name: str = 'default') BaseDatasource | FluentDatasource | LegacyDatasource#

Retrieve a given Datasource by name from the context's underlying DatasourceStore.

Parameters

datasource_name – The name of the target datasource.

Returns

The target datasource.

Raises

ValueError – The input datasource_name is None.

get_expectation_suite(expectation_suite_name: str | None = None, include_rendered_content: bool | None = None, ge_cloud_id: str | None = None) ExpectationSuite#

Get an Expectation Suite by name.

Parameters
  • expectation_suite_name (str) – The name of the Expectation Suite

  • include_rendered_content (bool) – Whether to re-populate rendered_content for each ExpectationConfiguration.

  • ge_cloud_id (str) –

    The GX Cloud ID for the Expectation Suite (unused)

    Deprecated since version 0.15.45.

Returns

An existing ExpectationSuite

Raises

DataContextError – There is no expectation suite with the name provided

get_profiler(name: str | None = None, ge_cloud_id: str | None = None, id: str | None = None) RuleBasedProfiler#

Retrieves a given Profiler by either name or id.

Parameters
  • name – The name of the target Profiler.

  • ge_cloud_id – The id associated with the target Profiler.

  • id

    The id associated with the target Profiler (preferred over ge_cloud_id).

    New in version 0.15.48: To be used in place of ge_cloud_id

Returns

The requested Profiler.

Raises

ProfilerNotFoundError – If the requested Profiler does not exists.

get_validator(datasource_name: Optional[str] = None, data_connector_name: Optional[str] = None, data_asset_name: Optional[str] = None, batch: Optional[Batch] = None, batch_list: Optional[List[Batch]] = None, batch_request: Optional[Union[BatchRequestBase, FluentBatchRequest]] = None, batch_request_list: Optional[List[BatchRequestBase]] = None, batch_data: Optional[Any] = None, data_connector_query: Optional[Union[IDDict, dict]] = None, batch_identifiers: Optional[dict] = None, limit: Optional[int] = None, index: Optional[Union[int, list, tuple, slice, str]] = None, custom_filter_function: Optional[Callable] = None, sampling_method: Optional[str] = None, sampling_kwargs: Optional[dict] = None, splitter_method: Optional[str] = None, splitter_kwargs: Optional[dict] = None, runtime_parameters: Optional[dict] = None, query: Optional[str] = None, path: Optional[str] = None, batch_filter_parameters: Optional[dict] = None, expectation_suite_ge_cloud_id: Optional[str] = None, batch_spec_passthrough: Optional[dict] = None, expectation_suite_name: Optional[str] = None, expectation_suite: Optional[ExpectationSuite] = None, create_expectation_suite_with_name: Optional[str] = None, include_rendered_content: Optional[bool] = None, expectation_suite_id: Optional[str] = None, **kwargs) Validator#

Retrieve a Validator with a batch list and an ExpectationSuite.

get_validator first calls get_batch_list to retrieve a batch list, then creates or retrieves an ExpectationSuite used to validate the Batches in the list.

Parameters
  • datasource_name – The name of the Datasource that defines the Data Asset to retrieve the batch for

  • data_connector_name – The Data Connector within the datasource for the Data Asset

  • data_asset_name – The name of the Data Asset within the Data Connector

  • batch – The Batch to use with the Validator

  • batch_list – The List of Batches to use with the Validator

  • batch_request – Encapsulates all the parameters used here to retrieve a BatchList. Use eitherbatch_request or the other params (but not both)

  • batch_request_list – A List of BatchRequest to use with the Validator

  • batch_data – Provides runtime data for the batch; is added as the key batch_data to the runtime_parameters dictionary of a BatchRequest

  • query – Provides runtime data for the batch; is added as the key query to the runtime_parameters dictionary of a BatchRequest

  • path – Provides runtime data for the batch; is added as the key path to the runtime_parameters dictionary of a BatchRequest

  • runtime_parameters – Specifies runtime parameters for the BatchRequest; can includes keys batch_data,query, and path

  • data_connector_query – Used to specify connector query parameters; specifically batch_filter_parameters,limit, index, and custom_filter_function

  • batch_identifiers – Any identifiers of batches for the BatchRequest

  • batch_filter_parameters – Filter parameters used in the data connector query

  • limit – Part of the data_connector_query, limits the number of batches in the batch list

  • index – Part of the data_connector_query, used to specify the index of which batch to return. Negative numbers retrieve from the end of the list (ex: -1 retrieves the last or latest batch)

  • custom_filter_function – A Callable function that accepts batch_identifiers and returns a bool

  • sampling_method – The method used to sample Batch data (see: Splitting and Sampling)

  • sampling_kwargs – Arguments for the sampling method

  • splitter_method – The method used to split the Data Asset into Batches

  • splitter_kwargs – Arguments for the splitting method

  • batch_spec_passthrough – Arguments specific to the ExecutionEngine that aid in Batch retrieval

  • expectation_suite_ge_cloud_id – The identifier of the ExpectationSuite to retrieve from the DataContext (can be used in place of expectation_suite_name)

  • expectation_suite_name – The name of the ExpectationSuite to retrieve from the DataContext

  • expectation_suite – The ExpectationSuite to use with the validator

  • create_expectation_suite_with_name – Creates a Validator with a new ExpectationSuite with the provided name

  • include_rendered_content – If True the ExpectationSuite will include rendered content when saved

  • **kwargs – Used to specify either batch_identifiers or batch_filter_parameters

Returns

A Validator with the specified Batch list and ExpectationSuite

Return type

Validator

Raises
  • DatasourceError – If the specified datasource_name does not exist in the DataContext

  • TypeError – If the specified types of the batch_request are not supported, or if thedatasource_name is not a str

  • ValueError – If more than one exclusive parameter is specified (ex: specifing more than one of batch_data, query or path), or if the ExpectationSuite cannot be created or retrieved using either the provided name or identifier

list_checkpoints() Union[List[str], List[great_expectations.data_context.types.resource_identifiers.ConfigurationIdentifier]]#

List existing Checkpoint identifiers on this context.

Returns

Either a list of strings or ConfigurationIdentifiers depending on the environment and context type.

list_datasources() List[dict]#

List the configurations of the datasources associated with this context.

Note that any sensitive values are obfuscated before being returned.

Returns

A list of dictionaries representing datasource configurations.

list_expectation_suite_names() List[str]#

Lists the available expectation suite names.

Returns

A list of suite names (sorted in alphabetic order).

run_checkpoint(checkpoint_name: str | None = None, ge_cloud_id: str | None = None, template_name: str | None = None, run_name_template: str | None = None, expectation_suite_name: str | None = None, batch_request: BatchRequestBase | FluentBatchRequest | dict | None = None, action_list: Sequence[ActionDict] | None = None, evaluation_parameters: dict | None = None, runtime_configuration: dict | None = None, validations: list[dict] | None = None, profilers: list[dict] | None = None, run_id: str | int | float | None = None, run_name: str | None = None, run_time: datetime.datetime | None = None, result_format: str | None = None, expectation_suite_ge_cloud_id: str | None = None, id: str | None = None, expectation_suite_id: str | None = None, **kwargs) CheckpointResult#

Validate using an existing Checkpoint.

Parameters
  • checkpoint_name – The name of a Checkpoint defined via the CLI or by manually creating a yml file

  • template_name – The name of a Checkpoint template to retrieve from the CheckpointStore

  • run_name_template – The template to use for run_name

  • expectation_suite_name – Expectation suite to be used by Checkpoint run

  • batch_request – Batch request to be used by Checkpoint run

  • action_list – List of actions to be performed by the Checkpoint

  • evaluation_parameters – $parameter_name syntax references to be evaluated at runtime

  • runtime_configuration – Runtime configuration override parameters

  • validations – Validations to be performed by the Checkpoint run

  • profilers – Profilers to be used by the Checkpoint run

  • run_id – The run_id for the validation; if None, a default value will be used

  • run_name – The run_name for the validation; if None, a default value will be used

  • run_time – The date/time of the run

  • result_format – One of several supported formatting directives for expectation validation results

  • ge_cloud_id – Great Expectations Cloud id for the checkpoint

  • expectation_suite_ge_cloud_id – Great Expectations Cloud id for the expectation suite

  • id

    Great Expectations Cloud id for the checkpoint (preferred over ge_cloud_id)

    New in version 0.15.48: To be used in place of ge_cloud_id

  • expectation_suite_id

    Great Expectations Cloud id for the expectation suite (preferred over expectation_suite_ge_cloud_id)

    New in version 0.15.48: To be used in place of expectation_suite_ge_cloud_id

  • **kwargs – Additional kwargs to pass to the validation operator

Returns

CheckpointResult

save_expectation_suite(expectation_suite: great_expectations.core.expectation_suite.ExpectationSuite, expectation_suite_name: Optional[str] = None, overwrite_existing: bool = True, include_rendered_content: Optional[bool] = None, **kwargs: Optional[dict]) None#

Save the provided ExpectationSuite into the DataContext using the configured ExpectationStore.

Deprecated since version 0.15.48: Part of the deprecated DataContext CRUD API

Parameters
  • expectation_suite – The ExpectationSuite to save.

  • expectation_suite_name – The name of this ExpectationSuite. If no name is provided, the name will be read from the suite.

  • overwrite_existing – Whether to overwrite the suite if it already exists.

  • include_rendered_content – Whether to save the prescriptive rendered content for each expectation.

  • kwargs – Additional parameters, unused

Returns

None

Raises

DataContextError – If a suite with the same name exists and overwrite_existing is set to False.

test_yaml_config(yaml_config: str, name: Optional[str] = None, class_name: Optional[str] = None, runtime_environment: Optional[dict] = None, pretty_print: bool = True, return_mode: Literal['instantiated_class', 'report_object'] = 'instantiated_class', shorten_tracebacks: bool = False)#

Convenience method for testing yaml configs.

test_yaml_config is a convenience method for configuring the moving parts of a Great Expectations deployment. It allows you to quickly test out configs for system components, especially Datasources, Checkpoints, and Stores.

For many deployments of Great Expectations, these components (plus Expectations) are the only ones you'll need.

test_yaml_config is mainly intended for use within notebooks and tests.

Relevant Documentation Links
Parameters
  • yaml_config – A string containing the yaml config to be tested

  • name – Optional name of the component to instantiate

  • class_name – Optional, overridden if provided in the config

  • runtime_environment – Optional override for config items

  • pretty_print – Determines whether to print human-readable output

  • return_mode – Determines what type of object test_yaml_config will return. Valid modes are "instantiated_class" and "report_object"

  • shorten_tracebacks – If true, catch any errors during instantiation and print only the last element of the traceback stack. This can be helpful for rapid iteration on configs in a notebook, because it can remove the need to scroll up and down a lot.

Returns

The instantiated component (e.g. a Datasource) OR a json object containing metadata from the component's self_check method. The returned object is determined by return_mode.

update_checkpoint(checkpoint: great_expectations.checkpoint.checkpoint.Checkpoint) great_expectations.checkpoint.checkpoint.Checkpoint#

Update a Checkpoint that already exists.

New in version 0.15.48.

Parameters

checkpoint – The checkpoint to use to update.

Raises

DataContextError – A suite with the given name does not already exist.

Returns

The updated Checkpoint.

update_datasource(datasource: BaseDatasource | FluentDatasource | LegacyDatasource, save_changes: bool | None = None) BaseDatasource | FluentDatasource | LegacyDatasource#

Updates a Datasource that already exists in the store.

Parameters
  • datasource – The Datasource object to update.

  • save_changes – do I save changes to disk?

Returns

The updated Datasource.

update_expectation_suite(expectation_suite: great_expectations.core.expectation_suite.ExpectationSuite) great_expectations.core.expectation_suite.ExpectationSuite#

Update an ExpectationSuite that already exists.

New in version 0.15.48.

Parameters

expectation_suite – The suite to use to update.

Raises

DataContextError – A suite with the given name does not already exist.

update_profiler(profiler: great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler) great_expectations.rule_based_profiler.rule_based_profiler.RuleBasedProfiler#

Update a Profiler that already exists.

New in version 0.15.48.

Parameters

profiler – The profiler to use to update.

Raises

ProfilerNotFoundError – A profiler with the given name/id does not already exist.

update_project_config(project_config: DataContextConfig | Mapping) DataContextConfig#

Update the context's config with the values from another config object.

Parameters

project_config – The config to use to update the context's internal state.

Returns

The updated project config.

view_validation_result(result: CheckpointResult) None#

New in version 0.16.15.

Opens a validation result in a browser.

Parameters

result – The result of a Checkpoint run.