autointent.metrics.retrieval.retrieval_mrr_intersecting#

autointent.metrics.retrieval.retrieval_mrr_intersecting(query_labels, candidates_labels, k=None)#

Calculate the Mean Reciprocal Rank (MRR) at position k for the intersecting labels.

MRR is calculated as:

$${\text{MRR@k}}_{\text{intersecting}}=\frac{1}{N}\sum _{i=1}^N\frac{1}{{\text{rank}}_i}$$

where:

• ${\text{rank}}_i$ is the rank position of the first relevant (intersecting) item in the top-k

results for query $i$, - $N$ is the total number of queries.

Intersecting relevance is determined by checking whether the query label intersects with the candidate labels.

Parameters:

• query_labels (autointent.metrics.custom_types.LABELS_VALUE_TYPE) – For each query, this list contains its class labels

• candidates_labels (autointent.metrics.custom_types.CANDIDATE_TYPE) – For each query, these lists contain class labels of items ranked by a retrieval model (from most to least relevant)

• k (int | None) – Number of top items to consider for each query

Returns:

Score of the retrieval metric

Return type:

float