How to configure an Expectation Store to use PostgreSQL
By default, new
ProfiledThe act of generating Metrics and candidate
Expectations from data.
ExpectationsA verifiable assertion about data.
are stored as
Expectation SuitesA collection of verifiable assertions about
data.
in JSON format in the
expectations/
subdirectory of your
great_expectations/
folder. Use the
information provided here to configure Great
Expectations to store Expectations in a PostgreSQL
database.
Prerequisites
- Completion of the Quickstart guide.
- A working installation of Great Expectations.
- A Data Context.
- An Expectations Suite.
- A PostgreSQL database with appropriate credentials.
1. Configure the
config_variables.yml
file with your
database credentials
GX recommends storing database credentials in the
config_variables.yml
file, which is
located in the uncommitted/
folder by
default, and not part of source control.
To add database credentials, open
config_variables.yml
and add the
following entry below the db_creds
key:
db_creds:
drivername: postgresql
host: '<your_host_name>'
port: '<your_port>'
username: '<your_username>'
password: '<your_password>'
database: '<your_database_name>'
To configure the
config_variables.yml
file, or additional
environment variables, see
How to configure credentials.
2. Identify your Data Context Expectations Store
Open great_expectations.yml
and find the
following entry:
expectations_store_name: expectations_store
stores:
expectations_store:
class_name: ExpectationsStore
store_backend:
class_name: TupleFilesystemStoreBackend
base_directory: expectations/
This configuration tells Great Expectations to look
for Expectations in the
expectations_store
Store. The default
base_directory
for
expectations_store
is
expectations/
.
3. Update your configuration file to include a new Store for Expectations
In the following example,
expectations_store_name
is set to
expectations_postgres_store
, but it can
be personalized. You also need to make some changes to
the store_backend
settings. The
class_name
is
DatabaseStoreBackend
, and
credentials
is
${db_creds}
to reference the
corresponding key in the
config_variables.yml
file.
expectations_store_name: expectations_postgres_store
stores:
expectations_postgres_store:
class_name: ExpectationsStore
store_backend:
class_name: DatabaseStoreBackend
credentials: ${db_creds}