Config Loaders

Load configuration from YAML, JSON, and TOML files.

See Also

• Config Overview - All config tools
• Schema - Config validation
• Registry - Component registry
• CLI - Command-line usage

---

datarax.config.loaders

TOML configuration loading and saving utilities.

This module provides functions for loading and saving TOML configuration files for Datarax pipelines and components.

logger module-attribute

logger = getLogger(__name__)

load_toml_from_path

load_toml_from_path(config_path: str | Path, encoding: str = 'utf-8') -> dict[str, Any]

Load a TOML configuration file.

Parameters:

Name	Type	Description	Default
config_path	str | Path	Path to the TOML configuration file	required
encoding	str	Character encoding to use (default: utf-8)	'utf-8'

Returns:

Type	Description
dict[str, Any]	Dictionary containing the parsed TOML configuration

Raises:

Type	Description
FileNotFoundError	If the configuration file does not exist
TOMLDecodeError	If the configuration file is invalid TOML

save_toml_to_path

save_toml_to_path(config: dict[str, Any], config_path: str | Path, encoding: str = 'utf-8') -> None

Save a configuration dictionary to a TOML file.

Parameters:

Name	Type	Description	Default
config	dict[str, Any]	Dictionary containing the configuration to save	required
config_path	str | Path	Path to save the TOML configuration file	required
encoding	str	Character encoding to use (default: utf-8)	'utf-8'

Raises:

Type	Description
OSError	If the file cannot be written

deep_merge_dict

deep_merge_dict(base: dict[str, Any], override: dict[str, Any]) -> dict[str, Any]

Recursively merge two dictionaries.

This function performs a deep merge of two dictionaries, with values from override taking precedence. For nested dictionaries, the merge is recursive.

Parameters:

Name	Type	Description	Default
base	dict[str, Any]	Base dictionary to merge into	required
override	dict[str, Any]	Dictionary with values that override the base	required

Returns:

Type	Description
dict[str, Any]	A new dictionary containing the merged values

load_config_from_path_with_includes

load_config_from_path_with_includes(config_path: str | Path, encoding: str = 'utf-8', include_key: str = 'include', processed_paths: set[Path] | None = None) -> dict[str, Any]

Load a TOML configuration file with support for including other files.

This function loads a TOML configuration file and processes any include directives to merge configurations from multiple files. Included files can themselves include other files, up to a reasonable depth.

Parameters:

Name	Type	Description	Default
config_path	str | Path	Path to the TOML configuration file	required
encoding	str	Character encoding to use (default: utf-8)	'utf-8'
include_key	str	Key that specifies included config files (default: "include")	'include'
processed_paths	set[Path] | None	Set of already processed paths to prevent cycles	None

Returns:

Type	Description
dict[str, Any]	Dictionary containing the merged TOML configuration

Raises:

Type	Description
FileNotFoundError	If any configuration file does not exist
RecursionError	If circular includes are detected
TOMLDecodeError	If any configuration file is invalid TOML