autointent.metrics.scoring.scoring_log_likelihood#

autointent.metrics.scoring.scoring_log_likelihood(labels, scores, eps=1e-10)#

Supports multiclass and multilabel cases.

Multiclass case: Mean negative cross-entropy for each utterance classification result:

\[\frac{1}{\ell}\sum_{i=1}^{\ell}\log(s[y[i]])\]

where s[y[i]] is the predicted score of the i-th utterance having the ground truth label.

Multilabel case: Mean negative binary cross-entropy:

\[\frac{1}{\ell}\sum_{i=1}^\ell\sum_{c=1}^C\Big[y[i,c]\cdot\log(s[i,c])+(1-y[i,c])\cdot\log(1-s[i,c])\Big]\]

where s[i,c] is the predicted score of the i-th utterance having the ground truth label c.

Parameters:

  • labels (autointent.metrics.custom_types.LABELS_VALUE_TYPE) – Ground truth labels for each utterance.

  • scores (autointent.metrics.custom_types.SCORES_VALUE_TYPE) – For each utterance, a list containing scores for each of n_classes classes.

  • eps (float) – A small value to avoid division by zero.

Returns:

Score of the scoring metric.

Return type:

float