@langwatch/scenario
    Preparing search index...

    Class ScenarioExecution

    Manages the execution of a single scenario.

    This class orchestrates the interaction between agents, executes the script, and manages the scenario's state. It also emits events that can be subscribed to for observing the scenario's progress.

    Note: This is an internal class. Most users will interact with the higher-level scenario.run() function instead of instantiating this class directly.

    import scenario from "@langwatch/scenario";

    // This is a simplified example of what `scenario.run` does internally.
    const result = await scenario.run({
      name: "My First Scenario",
      description: "A simple test of the agent's greeting.",
      agents: [
        scenario.userSimulatorAgent(),
        scenario.judgeAgent({
          criteria: ["Agent should respond with a greeting"],
        }),
      ],
      script: [
        scenario.user("Hello"),
        scenario.agent(),
        scenario.judge(),
      ]
    });

    console.log("Scenario result:", result.success);

    Implements

    Index

    Constructors

    constructor

    Properties

    events$

    Accessors

    messages threadId

    Methods

    addAgentTime agent execute fail hasResult judge message proceed setResult step succeed user

    Constructors

    Properties

    events$: Observable<
        | {
            batchRunId: string;
            metadata: { description?: string; name?: string };
            rawEvent?: any;
            scenarioId: string;
            scenarioRunId: string;
            scenarioSetId: string;
            timestamp: number;
            type: RUN_STARTED;
        }
        | {
            batchRunId: string;
            rawEvent?: any;
            results?: | null
            | {
                error?: string;
                metCriteria: string[];
                reasoning?: string;
                unmetCriteria: string[];
                verdict: Verdict;
            };
            scenarioId: string;
            scenarioRunId: string;
            scenarioSetId: string;
            status: ScenarioRunStatus;
            timestamp: number;
            type: RUN_FINISHED;
        }
        | {
            batchRunId: string;
            messages: (
                | { content: string; id: string; name?: string; role: "developer" }
                | { content: string; id: string; name?: string; role: "system" }
                | {
                    content?: string;
                    id: string;
                    name?: string;
                    role: "assistant";
                    toolCalls?: {
                        function: { arguments: string; name: string };
                        id: string;
                        type: "function";
                    }[];
                }
                | { content: string; id: string; name?: string; role: "user" }
                | { content: string; id: string; role: "tool"; toolCallId: string }
            )[];
            rawEvent?: any;
            scenarioId: string;
            scenarioRunId: string;
            scenarioSetId: string;
            timestamp: number;
            type: MESSAGE_SNAPSHOT;
        },
    > = ...

    An observable stream of events that occur during the scenario execution. Subscribe to this to monitor the progress of the scenario in real-time.

    Accessors

    Methods

    • Executes an agent turn. If content is provided, it's used as the agent's message. If not, the agent under test is called to generate a response. This is part of the ScenarioExecutionLike interface used by script steps.

      Parameters

      • Optionalcontent: string | CoreMessage

        The optional content of the agent's message.

      Returns Promise<void>

    • Executes the entire scenario from start to finish. This will run through the script and any automatic proceeding logic until a final result (success, failure, or error) is determined.

      Returns Promise<ScenarioResult>

      A promise that resolves with the final result of the scenario.

    • Invokes the judge agent to evaluate the current state of the conversation. This is part of the ScenarioExecutionLike interface used by script steps.

      Parameters

      • Optionalcontent: string | CoreMessage

        Optional message to pass to the judge.

      Returns Promise<null | ScenarioResult>

      A promise that resolves with the scenario result if the judge makes a final decision, otherwise null.

    • Lets the scenario proceed automatically for a specified number of turns. This simulates the natural flow of conversation between agents. This is part of the ScenarioExecutionLike interface used by script steps.

      Parameters

      • Optionalturns: number

        The number of turns to proceed. If undefined, runs until a conclusion or max turns is reached.

      • OptionalonTurn: (state: ScenarioExecutionStateLike) => void | Promise<void>

        A callback executed at the end of each turn.

      • OptionalonStep: (state: ScenarioExecutionStateLike) => void | Promise<void>

        A callback executed after each agent interaction.

      Returns Promise<null | ScenarioResult>

      A promise that resolves with the scenario result if a conclusion is reached.

    • Executes a single step in the scenario. A step usually corresponds to a single agent's turn. This method is useful for manually controlling the scenario's progress.

      Returns Promise<CoreMessage[] | ScenarioResult>

      A promise that resolves with the new messages added during the step, or a final scenario result if the step concludes the scenario.

    • Executes a user turn. If content is provided, it's used as the user's message. If not, the user simulator agent is called to generate a message. This is part of the ScenarioExecutionLike interface used by script steps.

      Parameters

      • Optionalcontent: string | CoreMessage

        The optional content of the user's message.

      Returns Promise<void>

    Settings

    Member Visibility

    On This Page

    Constructors
    Properties
    Accessors
    Methods