Database

The Database module provides classes for managing the context storage of a dialog system. The module can be used to store information such as the current state of the conversation and other data. This module includes the intermediate class (DBContextStorage) is a class that developers can inherit from in order to create their own context storage solutions. This class implements the basic functionality and can be extended to add additional features as needed.

class NameConfig

Bases: object

Configuration of names of different database parts, including table names, column names, field names, etc.

_main_table: Literal['main'] = 'main'

_turns_table: Literal['turns'] = 'turns'

_key_column: Literal['key'] = 'key'

_id_column: Literal['id'] = 'id'

_current_turn_id_column: Literal['current_turn_id'] = 'current_turn_id'

_created_at_column: Literal['created_at'] = 'created_at'

_updated_at_column: Literal['updated_at'] = 'updated_at'

_misc_column: Literal['misc'] = 'misc'

_framework_data_column: Literal['framework_data'] = 'framework_data'

_labels_field: Literal['labels'] = 'labels'

_requests_field: Literal['requests'] = 'requests'

_responses_field: Literal['responses'] = 'responses'

get_context_main_fields = ['current_turn_id', 'created_at', 'updated_at', 'misc', 'framework_data']

_lock(function)

class DBContextStorage(path, rewrite_existing=False, partial_read_config=None)

Bases: ABC

Base context storage class. Includes a set of methods for storing and reading different context parts.

Parameters:

• path (str) – Path to the storage instance.

• rewrite_existing (bool) – Whether TURNS modified locally should be updated in database or not.

• partial_read_config (Optional[Dict[Literal['labels', 'requests', 'responses'], Union[Literal['__all__'], int, Set[int]]]]) – Dictionary of subscripts for all possible turn items.

_default_subscript_value: int = 3

full_path

Full path to access the context storage, as it was provided by user.

path

full_path without a prefix defining db used.

rewrite_existing

Whether to rewrite existing data in the storage.

_subscripts

Subscripts control how many elements will be loaded from the database. Can be an integer, meaning the number of last elements to load. A special value for loading all the elements at once: “__all__”. Can also be a set of keys that should be loaded.

_sync_lock

Synchronization lock for the databases that don’t support asynchronous atomic reads and writes.

connected

Flag that marks if the storage is connected to the backend. Should be set in pipeline.run or later (lazily).

abstract property is_concurrent: bool

If the database backend support asynchronous IO.

classmethod _validate_field_name(field_name)

Return type:

str

abstract async _connect()

Return type:

None

async connect()

Connect to the backend context storage.

Return type:

None

abstract async _load_main_info(ctx_id)

Return type:

Optional[ContextMainInfo]

async load_main_info(ctx_id)

Load main information about the context.

Parameters:

ctx_id (str) – Context identifier.

Return type:

Optional[ContextMainInfo]

Returns:

Context main information (from MAIN table).

abstract async _update_context(ctx_id, ctx_info, field_info)

Return type:

None

async update_context(ctx_id, ctx_info=None, field_info=None)

Update context information.

Parameters:

• ctx_id (str) – Context identifier.

• ctx_info (Optional[ContextMainInfo]) – Context main information (will be written to MAIN table).

• field_info (Optional[List[Tuple[str, List[Tuple[int, bytes]], List[int]]]]) – Context turns information (will be written to TURNS table).

Return type:

None

abstract async _delete_context(ctx_id)

Return type:

None

async delete_context(ctx_id)

Delete context from context storage.

Parameters:

ctx_id (str) – Context identifier.

Return type:

None

abstract async _load_field_latest(ctx_id, field_name)

Return type:

List[Tuple[int, bytes]]

async load_field_latest(ctx_id, field_name)

Load the latest field data (specified by subscript value).

Parameters:

• ctx_id (str) – Context identifier.

• field_name (str) – Field name to load from TURNS table.

Return type:

List[Tuple[int, bytes]]

Returns:

List of tuples (step number, serialized value).

abstract async _load_field_keys(ctx_id, field_name)

Return type:

List[int]

async load_field_keys(ctx_id, field_name)

Load all field keys.

Parameters:

• ctx_id (str) – Context identifier.

• field_name (str) – Field name to load from TURNS table.

Return type:

List[int]

Returns:

List of all the step numbers.

abstract async _load_field_items(ctx_id, field_name, keys)

Return type:

List[Tuple[int, bytes]]

async load_field_items(ctx_id, field_name, keys)

Load field items (specified by key list). The items that are equal to None will be ignored.

Parameters:

• ctx_id (str) – Context identifier.

• field_name (str) – Field name to load from TURNS table.

• keys (List[int]) – List of keys to load.

Return type:

List[Tuple[int, bytes]]

Returns:

List of tuples (step number, serialized value).

abstract async _clear_all()

Return type:

None

async clear_all()

Clear all the chatsky tables and records.

Return type:

None

context_storage_factory(path, **kwargs)

Use context_storage_factory to lazy import context storage types and instantiate them. The function takes a database connection URI or its equivalent. It should be prefixed with database name, followed by the symbol triplet ‘://’.

Then, you should list the connection parameters like this: user:password@host:port/database The whole URI will then look like this:

• shelve://path_to_the_file/file_name

• json://path_to_the_file/file_name

• pickle://path_to_the_file/file_name

• sqlite+aiosqlite://path_to_the_file/file_name

• redis://:pass@localhost:6378/0

• mongodb://admin:pass@localhost:27016/admin

• mysql+asyncmy://root:pass@localhost:3306/test

• postgresql+asyncpg://postgres:pass@localhost:5430/test

• grpc://localhost:2134/local

• grpcs://localhost:2134/local

For context storages that write to local files, the function expects a file path instead of connection params: json://file.json When using sqlite backend your prefix should contain three slashes if you use Windows, or four in other cases: sqlite:////file.db

For MemoryContextStorage pass an empty string as path.

If you want to use additional parameters in class constructors, you can pass them to this function as kwargs.

Parameters:

path (str) – Path to the file.

Return type:

DBContextStorage