autointent.metrics.retrieval.RetrievalMetricFn#

class autointent.metrics.retrieval.RetrievalMetricFn#

Bases: Protocol

Protocol for retrieval metrics.

__call__(query_labels, candidates_labels, k=None)#

Calculate retrieval metric.

  • multiclass case: labels are integer

  • multilabel case: labels are binary

Parameters:

  • query_labels (autointent.metrics.custom_types.LABELS_VALUE_TYPE) – For each query, this list contains its class labels

  • candidates_labels (autointent.metrics.custom_types.CANDIDATE_TYPE) – For each query, these lists contain class labels of items ranked by a retrieval model (from most to least relevant)

  • k (int | None) – Number of top items to consider for each query

Returns:

Score of the retrieval metric

Return type:

float