autointent.modules.decision.JinoosDecision#

class autointent.modules.decision.JinoosDecision(search_space=None)#

Bases: autointent.modules.base.BaseDecision

Jinoos predictor module.

JinoosDecision predicts the best scores for single-label classification tasks and detects out-of-scope (OOS) samples based on a threshold.

Parameters:

search_space (list[autointent.custom_types.FloatFromZeroToOne] | None) – List of threshold values to search through for OOS detection

Examples:#

from autointent.modules import JinoosDecision
import numpy as np
scores = np.array([[0.2, 0.8], [0.6, 0.4], [0.1, 0.9]])
labels = [1, 0, 1]
search_space = [0.3, 0.5, 0.7]
predictor = JinoosDecision(search_space=search_space)
predictor.fit(scores, labels)
test_scores = np.array([[0.3, 0.7], [0.5, 0.5]])
predictions = predictor.predict(test_scores)
print(predictions)

[1, 0]
thresh: float#
name = 'jinoos'#

Name of the module.

supports_multilabel = False#

Whether the module supports multilabel classification

supports_multiclass = True#

Whether the module supports multiclass classification

supports_oos = True#

Whether the module supports oos data

search_space#
classmethod from_context(context, search_space=None)#

Initialize from context.

Parameters:

  • context (autointent.Context) – Context containing configurations and utilities

  • search_space (list[autointent.custom_types.FloatFromZeroToOne] | None) – List of threshold values to search through

Return type:

JinoosDecision

fit(scores, labels, tags=None)#

Fit the model.

Parameters:

  • scores (numpy.typing.NDArray[Any]) – Array of shape (n_samples, n_classes) with predicted scores

  • labels (autointent.custom_types.ListOfGenericLabels) – List of true labels

  • tags (list[autointent.schemas.Tag] | None) – List of Tag objects for mutually exclusive classes, or None

Return type:

None

predict(scores)#

Predict the best score.

Parameters:

scores (numpy.typing.NDArray[Any]) – Array of shape (n_samples, n_classes) with predicted scores

Returns:

List of predicted class indices or None for OOS samples

Raises:

MismatchNumClassesError – If the number of classes does not match the trained predictor

Return type:

list[int | None]

static jinoos_score(y_true, y_pred)#

Calculate Jinoos score.

The score is calculated as:

\[\frac{C_{in}}{N_{in}}+\frac{C_{oos}}{N_{oos}}\]

where \(C_{in}\) is the number of correctly predicted in-domain labels and \(N_{in}\) is the total number of in-domain labels. The same for OOS samples.

Parameters:

  • y_true (autointent.custom_types.ListOfGenericLabels) – True labels

  • y_pred (numpy.typing.NDArray[Any]) – Predicted labels

Returns:

Combined accuracy score for in-domain and OOS samples

Return type:

float