autointent.modules.abc.ScoringModule#

class autointent.modules.abc.ScoringModule#

Bases: autointent.modules.abc.Module, abc.ABC

Abstract base class for scoring modules.

Scoring modules predict scores for utterances and evaluate their performance using a scoring metric.

score(context, split, metric_fn)#

Evaluate the scorer on a test set and compute the specified metric.

Parameters:

context (autointent.Context) – Context containing test set and other data.
split (Literal['validation', 'test']) – Target split
metric_fn (autointent.metrics.ScoringMetricFn) – Function to compute the scoring metric.

Returns:

Computed metric value for the test set.

Return type:

float

get_assets()#

Retrieve assets generated during scoring.

Returns:: ScorerArtifact containing test scores and out-of-scope (OOS) scores.
Return type:: autointent.context.optimization_info.ScorerArtifact

abstract predict(utterances)#

Predict scores for a list of utterances.