Test & operate
Certify storage adapters
certifyDurableAdapters tests a candidate SubmissionLedger and ThreadStore pair and
returns one schema-encoded report.
| Level | Evidence |
|---|---|
| 1. Port contract | Runs all shared ledger and thread store conformance cases |
| 2. Coordinator convergence | Discovers six coordinator paths, injects reached failpoints and drives recovery |
| 3. Runtime loss | Exercises process termination or eviction, or records the committed suites that do |
Implement the store contract
Section titled “Implement the store contract”A ThreadStore materializes a thread, appends fenced batches, reads or observes
records, exports history, and inspects the tail. Appends must be atomic, digest-bound, idempotent by
batch ID, checked against the expected tail, and fenced by producer epoch. Reads decode stored
values through schemas.
Implement readPrompt as sparse, ascending pages through a captured canonical sequence, including
archived facts. Select Records.PROMPT_EVIDENCE_TAGS and decode with Records.PromptRecord, rather
than decoding full records and discarding fields. This read-only projection trusts write-time
validation and is never execution or integrity evidence. Bound raw hydration before decoding;
internal chunks must not shorten a page before its requested limit or captured range end.
Full read and snapshot-bound ThreadImportReader.read must preserve the canonical values used by
recovery and verification. Provide snapshot readers with ThreadImportReader.layer(...) so context
reconstruction uses that same transaction.
ThreadStore.checkpoints is optional storage for application projections. An adapter offering it
must run the generic checkpoint conformance suite. Retained-history execution does not use it.
A new ThreadCheckpoint needs only schemaVersion, threadId,
throughSequence, tailDigest, state, and createdAt. Adapters bind the sequence and digest
to a canonical batch tail; consumers decode state and decide whether its projection version
supports suffix replay. ThreadProjection uses version 3 and rejects older snapshots so
consumers can rebuild from canonical records.
Canonical execution progress is separate from application checkpoints. Implement native
RunContinuation reads for the newest owner record at or before a captured tail, bounded sorted
RunEvidence pages, and retained WorkHandoffs pages. Publish their indexes with the canonical
facts under the same epoch, expected-tail, and idempotency checks. Exact evidence references use
logical record identities and content digests; physical storage layout cannot change their meaning.
An append containing progress must reject conflicting revisions, changed original references, regressed accounting, or a frontier outside the same batch before writing. Preparations stay indexed after Run settlement; destination acknowledgement owns closure. An index grants no execution authority, and missing required evidence is a typed fault, not an empty completed Run. See Run continuations for recovery bounds.
This unreleased format accepts only fresh layout-21 storage and effect-agent/thread@3 archives.
Refuse predecessor, newer, or ambiguous layouts before any DDL or record mutation. Preserve the
rejected store; there is no layout upgrade or historical record decoder.
Thread age has no canonical record ceiling. Export and verification stream bounded pages. Keep each
ThreadRead page at or below 1,024 records and each CanonicalBatch at or below 256. An export
must preserve one captured tail and independent fact revision across its bounded pages, including
all immutable admissions, accepted commands, deliveries, and cross-range dependencies. Implement
exact native locators and streaming verification; never silently treat one page as complete history.
Implement SubmissionLedger.readAbortIntent as a strongly consistent read of one submission’s
abort intent. The runtime polls this method during execution, so its work must stay independent
of other submissions, approvals, and attached children. Unknown submissions fail with LedgerError.
The read grants no ownership, and canonicalRecordId must come from canonical history.
import type { SubmissionId } from "@yielded/agent/identifiers";import { AbortIntentRequest, SubmissionLedger } from "@yielded/agent/submission-ledger";import { Effect } from "effect";
const readAbort = Effect.fn(function* (submissionId: SubmissionId) { const ledger = yield* SubmissionLedger; return yield* ledger.readAbortIntent(AbortIntentRequest.make({ submissionId }));});Cloudflare keeps this read local to the submission’s owning Durable Object. The abort command still becomes canonical under the append gate before the runtime interrupts execution.
Joined input belongs to its host Submission. Validate a supplied RevertJoiningRequest.guard
atomically with rollback: leave a different host link untouched, and require the current host’s
ownership token unless that exact host has settled. The shared ledger conformance suite checks
these recovery boundaries.
Run the certification
Section titled “Run the certification”import { Effect } from "effect";import { certifyDurableAdapters } from "@yielded/agent-testing/certification";
const certificate = Effect.gen(function* () { return yield* certifyDurableAdapters({ adapter: { name: "@your-org/storage-yours" }, submissionLedger: yourLedgerLayer, threadStore: yourStoreLayer, tierThreeEvidence: ["path/to/your/real-loss.test.ts"], });});Provide Crypto.Crypto with NodeCrypto.layer on Node or BrowserCrypto.layer in workerd. Run
with a TestClock through @effect/vitest or a manual TestClock.layer() root. Tier 1 advances
leases, and tier 2 advances past the ownership lease during recovery. Pass
ownershipLeaseDuration when the ledger uses a different lease.
If both ports share a connection, pass the same combined layer instance to both fields. Layer memoization will acquire it once.
The returned CertificationReport includes adapter identity, the ledger durability claim, each
tier 1 result, each tier 2 cell, and tier 3 evidence. ok is true only when every executed check
passes. Statuses such as not-triggered, recorded-evidence, not-exercised, and
not-applicable describe scope. They count as neither a pass nor a failure.
Use fullyCertified when a gate requires complete durable-adapter certification in this run.
It requires ok: true, a durable adapter, and at least one passing real-loss case from crashLever.
All lever cases must belong to the real-loss suite. An empty lever reports not-exercised.
| Tier 3 status | ok when executed checks pass |
fullyCertified |
|---|---|---|
exercised with passing real-loss cases |
true |
true |
recorded-evidence |
true |
false |
not-exercised |
true |
false |
not-applicable for a non-durable adapter |
true |
false |
Any failed executed check makes both fields false. Recorded citations are external evidence;
the runner does not execute or verify those suites. Non-durable adapters can pass conformance
without earning durable certification. Reports keep the effect-agent/certification@2 format.
Interpret tier 2 results
Section titled “Interpret tier 2 results”Tier 2 first drives and verifies each of six scenarios without a fault, recording reached
coordinator locations. It then injects each reached failpoint into fresh thread state and
drives recovery through public operations. Locations observed during recovery also enter the
sweep. Each scenario/location pair is armed at most once. The runner resolves unknown outcomes
as SafeToRetry and approvals as approved only when explainThread authorizes that action.
Each cell reports:
convergedwhen the failpoint fired and recovery settled with verified invariants;not-triggeredwhen the location did not fire, including locations that share the scenario’s verified clean run because they were never reached;failedfor any other result, with bounded diagnostic detail.
not-triggered records the tested scope. It makes no fault-survival claim. Every scenario/location
pair remains in the report; shared clean results are identified in their detail. A location absent
from both discovery and the documented never-fired set is armed in every scenario, so new
failpoints cannot silently disappear from the sweep. Repository tests pin the fired paths per
scenario and the exact never-fired set. Dedicated suites and crash matrices cover those operator, abort, compaction,
background-worker, and Agent-update paths.
Those dedicated suites run separately; the certificate runner does not execute them.
The final invariant check recomputes the digest chain from EMPTY_TAIL_DIGEST and uses the same
checker as the administrative verify operation.
Provide tier 3 evidence
Section titled “Provide tier 3 evidence”@yielded/agent-storage-memoryreportsnot-applicablebecause it declares non-durable state.@yielded/agent-storage-sqliterecords the platform Node process-kill suites.@yielded/agent-storage-postgresreportsnot-exercisedbecause no committed process-kill suite drives it yet.@yielded/agent-storage-cloudflarerecords Durable Object eviction, cross-object subagent, and Miniflare restart suites.- A third-party adapter may pass
crashLeverto kill or evict its runtime and reopen storage for selected rows. Successful rows reportexercised. Without that lever or committed evidence, the report saysnot-exercised.
Import CertificationReport and certifyPorts from
@yielded/agent/testing/certification. The shared conformance cases live in
@yielded/agent/testing/thread-store-conformance and
@yielded/agent/testing/submission-ledger-conformance. Production schemas, ports,
replay, verification, and runtime APIs have their own public thread modules.
Certify subscription stores
Section titled “Certify subscription stores”An adapter that implements SubscriptionStore must also run
subscriptionStoreConformanceCases from @yielded/agent/testing/subscription-store-conformance. Give each case a fresh
partition. The cases cover intake cutoffs, deduplication, once selection, capacity, cancellation,
prepared recovery, catch-up, scan cursors, and replay after limits tighten.
The thread and submission certificate does not include these cases. Add restart or eviction tests around intake, partial fanout, selection, preparation, admission, and receipt persistence.