Filters

This module contains a collection of basic functions for history filtering to avoid cluttering LLMs context window.

class Return(value)

Bases: Enum

Enum that defines options for filtering turns.

NoReturn = 0

Do not include the turn.

Request = 1

Include request only.

Response = 2

Include response only.

Turn = 3

Include the entire turn (both request and response).

class BaseHistoryFilter(**data)

Bases: BaseModel, ABC

Base class for all message history filters.

abstract call(ctx, request, response, llm_model_name)

Decide whether to include request or response or both in the context history from a single turn.

The filter function is called repeatedly over all turns in context (up to history length limit in context_to_history()) to determine which parts of the turn to include.

Both request and response may be None. Even if such messages are not filtered out by this filter, they won’t be included in history.

Parameters:

• ctx (Context) – Context object.

• request (Optional[Message]) – Request message.

• response (Optional[Message]) – Response message.

• llm_model_name (str) – Name of the model that calls this filter in the Pipeline.models.

Return type:

Union[Return, int]

Returns:

Instance of Return enum or a corresponding int value.

__call__(ctx, request, response, llm_model_name)

Wrapper for call that catches exceptions and does not return any turn items if an exception occurs.

Parameters:

• ctx (Context) – Context object.

• request (Optional[Message]) – Request message.

• response (Optional[Message]) – Response message.

• llm_model_name (str) – Name of the model that calls this filter in the Pipeline.models.

Return type:

Return

Returns:

Instance of Return enum.

class MessageFilter(**data)

Bases: BaseHistoryFilter

Variant of history filter that allows to define simple filters that do not differentiate between requests and responses.

abstract single_message_filter_call(ctx, message, llm_model_name)

Determine based on a single message (which may be either request or response) whether to include the message in history.

Parameters:

• ctx (Context) – Context object.

• message (Optional[Message]) – Either request or response message.

• llm_model_name (str) – Name of the model that calls this filter in the Pipeline.models.

Return type:

bool

Returns:

Whether the message should be included in history.

call(ctx, request, response, llm_model_name)

Decide whether to include request or response or both in the context history from a single turn.

The filter function is called repeatedly over all turns in context (up to history length limit in context_to_history()) to determine which parts of the turn to include.

Both request and response may be None. Even if such messages are not filtered out by this filter, they won’t be included in history.

Parameters:

• ctx (Context) – Context object.

• request (Optional[Message]) – Request message.

• response (Optional[Message]) – Response message.

• llm_model_name (str) – Name of the model that calls this filter in the Pipeline.models.

Return type:

Union[Return, int]

Returns:

Instance of Return enum or a corresponding int value.

class DefaultFilter(**data)

Bases: BaseHistoryFilter

Filter used by default. Never filters out messages.

call(ctx, request, response, llm_model_name)

Decide whether to include request or response or both in the context history from a single turn.

The filter function is called repeatedly over all turns in context (up to history length limit in context_to_history()) to determine which parts of the turn to include.

Both request and response may be None. Even if such messages are not filtered out by this filter, they won’t be included in history.

Parameters:

• ctx (Context) – Context object.

• request (Optional[Message]) – Request message.

• response (Optional[Message]) – Response message.

• llm_model_name (str) – Name of the model that calls this filter in the Pipeline.models.

Return type:

Union[Return, int]

Returns:

Instance of Return enum or a corresponding int value.

class IsImportant(**data)

Bases: MessageFilter

Filter that checks if the “important” field in a Message.misc is True.

single_message_filter_call(ctx, message, llm_model_name)

Determine based on a single message (which may be either request or response) whether to include the message in history.

Parameters:

• ctx (Context) – Context object.

• message (Optional[Message]) – Either request or response message.

• llm_model_name (str) – Name of the model that calls this filter in the Pipeline.models.

Return type:

bool

Returns:

Whether the message should be included in history.

class FromModel(**data)

Bases: BaseHistoryFilter

Filter that checks if the response of the turn is generated by the currently

call(ctx, request, response, llm_model_name)

Decide whether to include request or response or both in the context history from a single turn.

The filter function is called repeatedly over all turns in context (up to history length limit in context_to_history()) to determine which parts of the turn to include.

Both request and response may be None. Even if such messages are not filtered out by this filter, they won’t be included in history.

Parameters:

• ctx (Context) – Context object.

• request (Optional[Message]) – Request message.

• response (Optional[Message]) – Response message.

• llm_model_name (str) – Name of the model that calls this filter in the Pipeline.models.

Return type:

Union[Return, int]

Returns:

Instance of Return enum or a corresponding int value.