autointent.metrics.RetrievalMetricFnWithOOS#

class autointent.metrics.RetrievalMetricFnWithOOS#

Bases: Protocol

Protocol for retrieval metrics that accept OOS-bearing query labels.

Returned by ignore_oos: the decorator filters out None entries from query_labels (and their paired candidate lists) before delegating to the underlying RetrievalMetricFn.

__call__(query_labels, candidates_labels, k=None)#

Calculate retrieval metric, dropping OOS-flagged queries first.

Parameters:

  • query_labels (autointent.custom_types.ListOfGenericLabels)

  • candidates_labels (autointent.metrics.custom_types.CANDIDATE_TYPE)

  • k (int | None)

Return type:

float