Skip to main content
Version: 0.14.13

How to use a Custom Expectation

Custom ExpectationsA verifiable assertion about data. are extensions to the core functionality of Great Expectations. Many Custom Expectations may be fit for a very specific purpose, or be at lower levels of stability and feature maturity than the core library.

As such, they are not available for use from the core library, and require registration and import to become available.

This guide will walk you through the process of utilizing Custom Expectations, whether they were built by you or came from the Great Expectations Experimental Library.

Prerequisites: This how-to guide assumes you have:

Steps​

1. File placement & import​

If you're using a Custom Expectation you've built, you'll need to place it in the great_expectations/plugins/expectations folder of your Great Expectations deployment.

When you instantiate your Data ContextThe primary entry point for a Great Expectations deployment, with configurations and methods for all supporting components., it will automatically make all plugins in the directory available for use, allowing you to import your Custom Expectation from that directory whenever and wherever it will be used. This import will be needed when an Expectation SuiteA collection of verifiable assertions about data. is created, and when a CheckpointThe primary means for validating data in a production deployment of Great Expectations. is defined and run.

2. Use in a Suite​

To use your Custom Expectation, we need to import it.

To do this, we first need to instantiate our Data Context.

For example, a pattern for importing a Custom Expectation ExpectColumnValuesToBeAlphabetical could look like: 

context = ge.get_context()
from expectations.expect_column_values_to_be_alphabetical import ExpectColumnValuesToBeAlphabetical

Now that your Custom Expectation has been imported, it is available with the same patterns as the core Expectations: 

validator.expect_column_values_to_be_alphabetical(column="test")

3. Use in a Checkpoint​

Once you have your Custom Expectation in a Suite, you will also need to make it available to your Checkpoint.

To do this, we'll need to put together our own Checkpoint script. From your command line, you can execute: 

great_expectations checkpoint new <my_checkpoint_name>

This will open a Jupyter Notebook allowing you to create a Checkpoint. If you would like to run your Checkpoint from this notebook, you will need to import your Custom Expectation again as above.

To continue to use this Checkpoint containing a Custom Expectation outside this notebook, we will need to set up a script for your Checkpoint.

To do this, execute the following from your command line: 

great_expectations checkpoint script <my_checkpoint_name>

This will create a script in your GE directory at great_expectations/uncommitted/run_my_checkpoint_name.py. That script can be edited that script to include the Custom Expectation import(s) you need: 

import sys

from great_expectations.checkpoint.types.checkpoint_result import CheckpointResult
from great_expectations.data_context import DataContext

data_context: DataContext = DataContext(
    context_root_dir="/your/path/to/great_expectations"
)

from expectations.expect_column_values_to_be_alphabetical import ExpectColumnValuesToBeAlphabetical

result: CheckpointResult = data_context.run_checkpoint(
    checkpoint_name="my_checkpoint_name",
    batch_request=None,
    run_name=None,
)

if not result["success"]:
    print("Validation failed!")
    sys.exit(1)

print("Validation succeeded!")
sys.exit(0)

The Checkpoint can then be run with:

python great_expectations/uncommitted/run_my_checkpoint_name.py

Congratulations!
🎉 You've just run a Checkpoint with a Custom Expectation! 🎉