Skip to main content
Version: 0.17.23

ConfiguredAssetDBFSDataConnector

class great_expectations.datasource.data_connector.ConfiguredAssetDBFSDataConnector(name: str, datasource_name: str, base_directory: str, assets: dict, execution_engine: great_expectations.execution_engine.execution_engine.ExecutionEngine, default_regex: Optional[dict] = None, glob_directive: str = '**/*', sorters: Optional[list] = None, batch_spec_passthrough: Optional[dict] = None, id: Optional[str] = None)#

Extension of ConfiguredAssetFilesystemDataConnector used to connect to the DataBricks File System (DBFS).

Being a Configured Asset Data Connector, it requires an explicit list of each Data Asset it can connect to. While this allows for fine-grained control over which Data Assets may be accessed, it requires more setup.

Parameters:
  • name (str) – required name for DataConnector

  • datasource_name (str) – required name for datasource

  • assets (dict) – dict of asset configuration (required for ConfiguredAssetDataConnector). These can each have their own regex and sorters

  • execution_engine (ExecutionEngine) – Reference to ExecutionEngine

  • default_regex (dict) – optional regex configuration for filtering data_references

  • glob_directive (str) – glob for selecting files in directory (defaults to *)

  • sorters (list) – optional list of sorters for sorting data_references

  • batch_spec_passthrough (dict) – dictionary with keys that will be added directly to batch_spec

get_available_data_asset_names() List[str]#

Return the list of asset names known by this DataConnector.

Returns:

A list of available names