Logging to stdout and file

This guide will teach you how to configure logging in AutoIntent. By default, it is fully disabled.

It will be demonstrated on toy search_space example:

from pathlib import Path

from autointent import Dataset, Pipeline
from autointent.configs import LoggingConfig

search_space = [
    {
        "node_type": "scoring",
        "target_metric": "scoring_roc_auc",
        "search_space": [
            {
                "module_name": "knn",
                "k": [1],
                "weights": ["uniform"],
                "embedder_config": ["avsolatorio/GIST-small-Embedding-v0"],
            },
        ],
    },
    {
        "node_type": "decision",
        "target_metric": "decision_accuracy",
        "search_space": [
            {"module_name": "threshold", "thresh": [0.5]},
            {"module_name": "argmax"},
        ],
    },
]

log_config = LoggingConfig(project_dir=Path("logging_tutorial"))
pipeline_optimizer = Pipeline.from_search_space(search_space)
pipeline_optimizer.set_config(log_config)

dataset = Dataset.from_hub("AutoIntent/clinc150_subset")

/home/runner/.cache/pypoetry/virtualenvs/autointent-FDypUDHQ-py3.10/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html
  from .autonotebook import tqdm as notebook_tqdm

Fully Custom Logging

One can fully customize logging via python’s standard module `logging <https://docs.python.org/3/library/logging.html>`__. Everything you need to do is configure it before AutoIntent execution:

import logging

logging.basicConfig(level="INFO")
pipeline_optimizer.fit(dataset)

INFO:autointent._pipeline._pipeline:starting pipeline optimization...
INFO:autointent.nodes._node_optimizer:Starting NodeType.scoring node optimization...
/home/runner/work/AutoIntent/AutoIntent/autointent/nodes/_node_optimizer.py:99: ExperimentalWarning: BruteForceSampler is experimental (supported from v3.1.0). The interface can change in the future.
  sampler_instance = optuna.samplers.BruteForceSampler(seed=context.seed)  # type: ignore[assignment]
[I 2025-03-08 22:26:47,341] A new study created in memory with name: no-name-e25c4a62-8952-45e9-8901-b8556233be38
INFO:sentence_transformers.SentenceTransformer:Use pytorch device_name: cpu
INFO:sentence_transformers.SentenceTransformer:Load pretrained SentenceTransformer: avsolatorio/GIST-small-Embedding-v0
Batches: 100%|██████████| 1/1 [00:00<00:00,  9.48it/s]
Batches: 100%|██████████| 1/1 [00:00<00:00, 35.92it/s]
Batches: 100%|██████████| 1/1 [00:02<00:00,  2.22s/it]
Batches: 100%|██████████| 1/1 [00:00<00:00,  1.72it/s]
INFO:autointent.nodes._node_optimizer:NodeType.scoring node optimization is finished!
INFO:autointent.nodes._node_optimizer:Starting NodeType.decision node optimization...
/home/runner/work/AutoIntent/AutoIntent/autointent/nodes/_node_optimizer.py:99: ExperimentalWarning: BruteForceSampler is experimental (supported from v3.1.0). The interface can change in the future.
  sampler_instance = optuna.samplers.BruteForceSampler(seed=context.seed)  # type: ignore[assignment]
WARNING:autointent.modules.base._base:"argmax" is NOT designed to handle OOS samples, but your data contains it. So, using this method reduces the power of classification.
INFO:autointent.nodes._node_optimizer:NodeType.decision node optimization is finished!
Batches: 100%|██████████| 1/1 [00:01<00:00,  1.52s/it]
/home/runner/.cache/pypoetry/virtualenvs/autointent-FDypUDHQ-py3.10/lib/python3.10/site-packages/sklearn/metrics/_classification.py:1565: UndefinedMetricWarning: Precision is ill-defined and being set to 0.0 in labels with no predicted samples. Use `zero_division` parameter to control this behavior.
  _warn_prf(average, modifier, f"{metric.capitalize()} is", len(result))

<autointent.context._context.Context at 0x7f28999e3bb0>

See external tutorials and guides about logging module.

Export from AutoIntent

If you don’t have to customize logging, you can export our configuration. Everything you need to do is setup it before AutoIntent execution:

from autointent import setup_logging

setup_logging("INFO", log_filename="tests/logs/my_exp")

"""
The first parameter affects the logs to the standard output stream. The second parameter is optional. If it is specified, then the "DEBUG" messages are logged to the file, regardless of what is specified by the first parameter.
"""

'\nThe first parameter affects the logs to the standard output stream. The second parameter is optional. If it is specified, then the "DEBUG" messages are logged to the file, regardless of what is specified by the first parameter.\n'