autointent.metrics.retrieval_ndcg_intersecting#

autointent.metrics.retrieval_ndcg_intersecting(query_labels, candidates_labels, k=None)#

Calculate the Normalized Discounted Cumulative Gain (NDCG) at position k for the intersecting labels.

NDCG at position \(k\) for intersecting labels is calculated as:

\[\text{NDCG@k}_{\text{intersecting}} = \frac{\text{DCG@k}_{\text{intersecting}}} {\text{IDCG@k}_{\text{intersecting}}}\]

where:

  • \(\text{DCG@k}_{\text{intersecting}}\) is the Discounted Cumulative Gain for the intersecting labels at position \(k\),

  • \(\text{IDCG@k}_{\text{intersecting}}\) is the Ideal Discounted Cumulative Gain for the intersecting labels at position \(k\).

Intersecting relevance is determined by checking whether the query labels overlap with the candidate labels. NDCG values are normalized between 0 and 1, where 1 indicates the ideal ranking.

Parameters:

  • query_labels (autointent.metrics.custom_types.LABELS_VALUE_TYPE) – For each query, this list contains its class labels

  • candidates_labels (autointent.metrics.custom_types.CANDIDATE_TYPE) – For each query, these lists contain class labels of items ranked by a retrieval model (from most to least relevant)

  • k (int | None) – Number of top items to consider for each query

Returns:

Score of the retrieval metric

Return type:

float