Skip to main content
Version: 0.16.16

Store

In progress

The Great Expectations CLI is no longer the preferred method for implementing and configuring Great Expectations. This topic will be updated soon to reflect this change. For more information, see A fond farewell to the CLI.

A Store is a connector to store and retrieve information about metadata in Great Expectations. Stores are available from your Data Context.

Great Expectations supports a variety of Stores for different purposes, but the most common Stores are Expectation Stores, Validations Stores, Checkpoint Stores, Metric Stores, and Evaluation Parameter Stores. Data Docs Stores can also be configured for Data Doc Sites. Each of these Stores is tailored to a specific type of information.

  • Expectation Store: a connector to store and retrieve information about collections of verifiable assertions about data. These are Stores for Expectation Suites.
  • Validation Result Store: a connector to store and retrieve information about objects generated when data is Validated against an Expectation Suite.
  • Checkpoint Store: a connector to store and retrieve information about means for validating data in a production deployment of Great Expectations.
  • Evaluation Parameter Store: a connector to store and retrieve information about parameters used during Validation of an Expectation which reference simple expressions or previously generated Metrics.
  • Data Docs Store: a connector to store and retrieve information pertaining to Human readable documentation generated from Great Expectations metadata detailing Expectations, Validation Results, etc.
  • Metric Stores: a connector to store and retrieve information about computed attributes of data, such as the mean of a column. These differ from the Evaluation Parameter Store in how they are formatted. The data in a Metrics Store is intended to be used for generating reports and analyzing trends, rather than as Evaluation Parameter values.

Relationship to other objects​

Each type of Store is designed to interact with a specific subset of information, and thus interacts with a specific subset of objects in Great Expectations. However, all Stores can be listed and modified through your Data Context. For further information on how a given type of Store relates to other objects, please see the corresponding Store type's technical term page:

Use cases​

All Stores are configured during the Setup step of working with Great Expectations. For the most part, the default configurations will permit you to work with Great Expectations in a local environment. You can, however, configure your Stores to be hosted elsewhere.

When creating Expectations, you will use Expectation Stores to store your Expectation SuitesA collection of verifiable assertions about data.. You may also use Validation Result Stores or Evaluation Parameter Stores to configure some ExpectationsA verifiable assertion about data. that require Evaluation ParametersA dynamic value used during Validation of an Expectation which is populated by evaluating simple expressions or by referencing previously generated metrics. as part of their definition, provided you have previously stored relevant MetricsA computed attribute of data such as the mean of a column. in those Stores and then come back to create a new Expectation Suite that references them.

When Validating data, you may store new CheckpointsThe primary means for validating data in a production deployment of Great Expectations. (or retrieve existing ones) from your Checkpoint Store. Checkpoints may also use Expectation Suites retrieved by an Expectation Store, and Expectations that are run by a Checkpoint may retrieve Metrics as input (Evaluation Parameters) from Validation ResultsGenerated when data is Validated against an Expectation or Expectation Suite. in a Validation Results Store, or from values stored in an Evaluation Parameter Store. Checkpoints may also write information about Validation Results into a Validation Results Store, or information about Metrics into an Evaluation Parameter Store (which is why they are also known as Metric Stores).

Access​

The stores key in your great_expectations.yml file lists your available Stores.

The data_docs_sites key in your great_expectations.yml file defines your Data Docs Store configurations. store_backend keys below data_docs_sites identify the Data Doc Store used by the Data Docs site.

Run the following Python command to view your Stores: 

import great_expectations as gx

context = gx.get_context()
context.stores

Data Docs Stores are the one exception to the above. They are instead configured under the data_docs_sites key in the great_expectations.yml file. Each entry under data_docs_sites will have a store_backend key. The information under store_backend will correspond to the Data Doc Store used by that site. Data Doc Stores are not listed by the great_expectations store list command.

Configure​

Your Stores configurations are located in your great_expectations.yml file below the stores key. Three stores are required: an Expectation Store, a Validation Result Store, and an Evaluation Parameter Store must always have a valid entry in the great_expectations.yml file. Additional Stores can be configured for uses such as Data Docs and Checkpoints (and a Checkpoint Store configuration entry will automatically be added if one does not exist when you attempt to save a Checkpoint for the first time.)

Note that unlike other Stores, your Data Docs Stores are configured under each individual site under the data_docs_sites key in the great_expectations.yml file.

For more information about a specific type of Store, see the Store type's technical term definition page: