autointent.context.optimization_info.OptimizationInfo#

class autointent.context.optimization_info.OptimizationInfo#

Tracks optimization results, including trials, artifacts, and modules.

This class provides methods for logging optimization results, retrieving the best-performing modules and artifacts, and generating configuration for inference nodes.

artifacts#

Container for storing optimization artifacts.

trials#

Container for storing optimization trials.

modules#

Container for storing module instances.

pipeline_metrics#

Dictionary storing pipeline-level metrics.

artifacts#
trials#
modules#
pipeline_metrics: dict[str, float]#
log_module_optimization(node_type, module_name, module_params, metric_value, metric_name, metrics, artifact, module_dump_dir, module=None)#

Log optimization results for a module.

Parameters:

  • node_type (str) – Type of the node being optimized.

  • module_name (str) – Type of the module.

  • module_params (dict[str, Any]) – Parameters of the module for the trial.

  • metric_value (float) – Metric value achieved by the module.

  • metric_name (str) – Name of the evaluation metric.

  • metrics (dict[str, float]) – Dictionary of metric names and their values.

  • artifact (autointent.context.optimization_info._data_models.Artifact) – Artifact generated by the module.

  • module_dump_dir (str | None) – Directory where the module is dumped.

  • module (BaseModule | None) – The module instance, if available.

Return type:

None

get_best_embedder()#

Retrieve the name of the best embedder from the retriever node.

Returns:

Configuration of the best embedder.

Return type:

autointent.configs.EmbedderConfig

get_best_train_scores()#

Retrieve the train scores from the best scorer node.

Returns:

Train scores as a numpy array.

Return type:

numpy.typing.NDArray[numpy.float64] | None

get_best_validation_scores()#

Retrieve the validation scores from the best scorer node.

Returns:

Validation scores as a numpy array.

Return type:

numpy.typing.NDArray[numpy.float64] | None

get_best_folded_scores()#

Retrieve the validation scores from the best scorer node.

Returns:

Validation scores as a numpy array.

Return type:

list[numpy.typing.NDArray[numpy.float64]] | None

get_best_test_scores()#

Retrieve the test scores from the best scorer node.

Returns:

Test scores as a numpy array.

Return type:

numpy.typing.NDArray[numpy.float64] | None

dump_evaluation_results()#

Dump evaluation results for all nodes.

Returns:

Dictionary containing metrics and configurations for all nodes.

Return type:

dict[str, Any]

get_inference_nodes_config(asdict=False)#

Generate configuration for inference nodes based on the best trials.

Parameters:

asdict (bool) – Whether to return the configuration as dictionaries.

Returns:

List of configurations for inference nodes.

Return type:

list[autointent.configs.InferenceNodeConfig]

get_best_modules()#

Retrieve the best modules for all node types.

Returns:

Dictionary of the best modules for each node type.

Return type:

dict[autointent.custom_types.NodeType, autointent.modules.base.BaseModule]