dialog2graph.pipelines.core#

Submodules#

Classes#

RecursiveDialogSampler

Recursive dialog sampler for the graph

BasePipeline

Base class for pipelines

Package Contents#

class dialog2graph.pipelines.core.RecursiveDialogSampler[source]#

Bases: dialog2graph.pipelines.core.algorithms.DialogGenerator

Recursive dialog sampler for the graph

invoke(graph: dialog2graph.pipelines.core.graph.BaseGraph, cycle_ends_model: langchain_core.language_models.chat_models.BaseChatModel, upper_limit: int, sampling_max: int = 5000000) → list[dialog2graph.pipelines.core.dialog.Dialog][source]#

Extract all the dialogs from the graph

Parameters:

  • graph (dialog2graph.pipelines.core.graph.BaseGraph) – used to extract dialogs from it

  • cycle_ends_model (langchain_core.language_models.chat_models.BaseChatModel) – LLM(BaseChatModel) to find cycling ends of the graph

  • upper_limit (int) – limits from above repeats_limit used in recursive get_dialogs method

  • sampling_max (int) – maximum number of found dialogs

Returns:

list of dialogs

Raises:

ValueError – “Not all utterances present” if match_dg_triplets returns False

Return type:

list[dialog2graph.pipelines.core.dialog.Dialog]

async ainvoke(*args, **kwargs)[source]#
async evaluate(graph, upper_limit, target_dialogs, report_type=Literal['dict', 'dataframe'])[source]#
class dialog2graph.pipelines.core.BasePipeline(/, **data: Any)[source]#

Bases: pydantic.BaseModel

Base class for pipelines .. attribute:: model_storage

An object to manage and store models used in the pipeline.

type:

ModelStorage

Parameters:

data (Any)

sim_model#

The key for the similarity embedder model in the model storage.

Type:

str

model_storage: dialog2graph.pipelines.model_storage.ModelStorage = None#
sim_model: str = None#
name: str = None#
steps: list[dialog2graph.pipelines.core.algorithms.DialogGenerator | dialog2graph.pipelines.core.algorithms.DialogAugmentation | dialog2graph.pipelines.core.algorithms.GraphGenerator | dialog2graph.pipelines.core.algorithms.GraphExtender] = None#
invoke(raw_data: dialog2graph.pipelines.helpers.parse_data.PipelineRawDataType, enable_evals=False) → Tuple[Any, dialog2graph.pipelines.report.PipelineReport][source]#

Invoke the pipeline to process the raw data and generate a report.

This method processes the given raw data through each step in the pipeline, generating both output data (result of the pipeline) and a report detailing the pipeline’s execution. It measures execution time, performs simple graph comparisons, and optionally evaluates the results with more detailed comparisons.

Parameters:

  • raw_data (PipelineRawDataType) – The raw input data to be processed by the pipeline.

  • enable_evals (bool, optional) – If True, performs additional evaluations and adds more detailed comparisons to the report.

Returns:

A tuple containing the final output of the pipeline

and a detailed report of the pipeline’s execution.

Return type:

Tuple[Any, PipelineReport]