autointent.metrics.retrieval_mrr#

autointent.metrics.retrieval_mrr(query_labels, candidates_labels, k=None)#

Calculate the Mean Reciprocal Rank (MRR) at position k.

MRR is calculated as:

\[\text{MRR@k} = \frac{1}{N} \sum_{i=1}^N \frac{1}{\text{rank}_i}\]

where: - \(\text{rank}_i\) is the rank position of the first relevant item in the top-k results for query \(i\), - \(N\) is the total number of queries.

Parameters:

  • query_labels (autointent.metrics.custom_types.LABELS_VALUE_TYPE) – For each query, this list contains its class labels

  • candidates_labels (autointent.metrics.custom_types.CANDIDATE_TYPE) – For each query, these lists contain class labels of items ranked by a retrieval model (from most to least relevant)

  • k (int | None) – Number of top items to consider for each query

Returns:

Score of the retrieval metric

Return type:

float

On this page