Deterministic testing

@yielded/agent-testing provides a deterministic Effect AI LanguageModel layer. Use it to test the real interpreter without network access, credentials, provider latency, or model variance.

Provide a scripted model layer

import { Model } from "effect/ai";
import { ScriptedModel } from "@yielded/agent-testing/scripted-model";

const TestModel = Model.make("scripted", "test-model", ScriptedModel.layer(turns));

A script can emit text, reasoning, tool calls, usage, malformed sequences, typed model failures, or a stream that waits for interruption. Hooks can inspect the normalized Prompt and detect stream finalization.

Choose a testing entry point

The package root exports the ScriptedModel module namespace. Import its service, request, turn, hook types, and schemas directly from @yielded/agent-testing/scripted-model. Specialized helpers have separate paths:

Import	Contents
@yielded/agent-testing/certification	Durable adapter certification
@yielded/agent-testing/chaos	Seeded plans and convergence checks
@yielded/agent-testing/code-executor-conformance	CodeExecutor adapter conformance
@yielded/agent-testing/code-executor-substitute	Deterministic in-process executor substitute
@yielded/agent-testing/travel-planner	Travel Planner definitions, services, and scenarios
@yielded/agent-testing/docs-researcher	Docs Researcher definitions and MCP delegation fixtures

These paths ship JavaScript and declarations. Install the storage and runtime adapters used by your tests directly.

Mutable runtime failpoint controls live in @yielded/agent/testing/durable-failpoint-test-control. Its .layer provides the production failpoint service and mutable test control over one Ref. @yielded/agent-storage-cloudflare/testing/do-storage-failpoint-testing exports evictionFailpointHandler.

Exercise the public runtime

Provide the same layers as the application. Override the default IdGenerator reference with a deterministic counter when assertions depend on stable IDs.

import { Effect, Layer, Ref, Schema } from "effect";
import { InMemory } from "@yielded/agent";
import { ThreadId, RunId, TurnId } from "@yielded/agent/identifiers";
import { IdGenerator } from "@yielded/agent/id-generator";

const DeterministicIdGeneratorLive = Layer.effect(
  IdGenerator,
  Effect.gen(function* () {
    const sequence = yield* Ref.make(0);
    const next = Ref.updateAndGet(sequence, (n) => n + 1);

    return IdGenerator.of({
      nextThreadId: next.pipe(Effect.map((n) => Schema.decodeSync(ThreadId)(`thread-${n}`))),
      nextRunId: next.pipe(Effect.map((n) => Schema.decodeSync(RunId)(`run-${n}`))),
      nextTurnId: next.pipe(Effect.map((n) => Schema.decodeSync(TurnId)(`turn-${n}`))),
    });
  }),
);

const TestRuntimeLive = Layer.mergeAll(
  InMemory.layer,
  ToolkitLive,
  DomainServicesTest,
  DeterministicIdGeneratorLive,
);

it.effect("commits tool results in declaration order", () =>
  AgentRuntime.run(Definition, input).pipe(
    Effect.provide(TestModel),
    Effect.provide(TestRuntimeLive),
    Effect.tap((result) => Effect.sync(() => expect(result.output).toEqual(expected))),
  ),
);

This runs the public definition, binding, toolkit handlers, scheduler, stream reducer, and output decoder. Completed run calls need no extra caller Scope. Assert run-local finalizers after the call or interrupted fiber finishes. Keep a Scope around start and resources acquired by the test.

Assert behavior beyond output

Useful assertions include:

• normalized Prompt content and available tool names;
• parameter decoding and typed failures;
• the complete semantic RunEvent trace;
• actual completion order and committed declaration order;
• policy exhaustion and approval suspension;
• interruption and finalizers;
• inferred Effect<A, E, R> types.

Test storage contracts, not implementations

Run shared conformance cases against every memory, SQLite, or custom store. The thread cases cover materialization, idempotent append, tail conflicts, fencing, observation offsets, export, and corruption. Stores with ThreadStore.checkpoints also run threadCheckpointConformanceCases.

Durable adapters also need ledger conformance, failpoint coverage, and real process-loss tests. See Certify storage adapters.