Skip to content

Blackbox Testing

Testing via public interfaces

Blackbox testing examines software functionality without inspecting internal implementation. Learn more about blackbox testing.

For Scenario Users

In Scenario, blackbox testing means your adapter calls the agent's public interface—the same HTTP endpoint, CLI command, or SDK method that users interact with—rather than directly instantiating agent classes or mocking internal components.

This tests your agent as it runs in production: with real databases, authentication, middleware, and infrastructure.

Example

typescript
import scenario, { AgentAdapter, AgentRole } from "@langwatch/scenario";
 
// ✅ Blackbox: adapter calls real HTTP endpoint (no mocking)
const adapter: AgentAdapter = {
  role: AgentRole.AGENT,
  call: async (input) => {
    // Make real HTTP request to actual agent server
    return await fetch("http://localhost:3000/chat", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ messages: input.messages }),
    })
      .then((r) => r.json())
      .then((r) => r.response);
  },
};
 
// Scenario tests the complete production stack
await scenario.run({
  agents: [scenario.userSimulatorAgent(), adapter],
  // ... scenario configuration
});

Compare with mocked testing where you might mock tool functions or bypass HTTP entirely. Blackbox tests the complete stack.

When to Use

  • Production validation: Verify deployed agent works correctly
  • Integration testing: Test all components together (server, database, agent)
  • API contracts: Ensure public interface doesn't break

See Also