Skip to content

Build an agent

Context management

A Thread can outlive a model’s context window. Keep three things separate: retained conversation history, the working prompt for one model call, and application-owned memory shared across tasks. Compaction reduces the prompt; it does not erase history or resolve unfinished work.

History, context, and memory. The runtime builds a bounded model prompt from Thread history and authorized application memory. Compaction changes the prompt view, not the retained history.
  • Yielded Agent: Thread historyRetained conversation
  • Your code: Memory sourcesAuthorized reference material
  • Yielded Agent: Working promptPrepared and bounded per call
  • External: Model
  • Thread history to Working prompt: compact
  • Memory sources to Working prompt: recall
  • Working prompt to Model
import {
class AgentPolicy

Schema class for agent run limits; wall-clock duration remains finite.

AgentPolicy
,
class CompactionPolicy

How the engine reduces model context when contextTokenLimit is crossed.

CompactionPolicy
} from "@yielded/agent/agent-policy";
const
const policy: AgentPolicy
policy
=
class AgentPolicy

Schema class for agent run limits; wall-clock duration remains finite.

AgentPolicy
.
AgentPolicy.resolve(input?: Partial<AgentPolicyInput>, parent?: AgentPolicy): AgentPolicy

Resolve an optional declaration against standalone or inherited defaults.

resolve
({
contextTokenLimit?: number | undefined

Per-model-call live-context bound; crossing it triggers compaction rather than failure.

contextTokenLimit
: 40_000,
tokenBudget?: number | undefined

Observed cumulative usage can cross this threshold before exhaustion is handled.

tokenBudget
: 200_000,
completionReserveTokens?: number | undefined

Tokens reserved for final delivery; defaults to 20% of tokenBudget, capped at 4,096.

completionReserveTokens
: 16_000,
compaction?: CompactionPolicy | undefined

Compaction strategy applied when contextTokenLimit is crossed; defaults documented on CompactionPolicy.

compaction
:
class CompactionPolicy

How the engine reduces model context when contextTokenLimit is crossed.

CompactionPolicy
.
CompactionPolicy.make(input?: CompactionPolicyInput): CompactionPolicy

Normalize and validate compaction bounds, filling documented defaults.

make
({
keepRecentTokens?: number | undefined

Estimated tokens of recent history preserved verbatim across a compaction; defaults to 20_000.

keepRecentTokens
: 8_000 }),
});

Use this policy on an Agent with a model whose window leaves room for output and request overhead. These numbers are workload choices, not provider defaults. contextTokenLimit bounds one call; tokenBudget counts cumulative input and output. completionReserveTokens leaves cumulative budget for finishing. It is separate from a model’s per-call output allowance.

Image FileParts are counted automatically from image dimensions when available, with a fixed reservation for remote or unreadable images. Bytes, base64, and data URLs are image transport, not prompt text. These are context estimates; provider usage is authoritative. A resolved model’s estimateMessageTokens can refine them. Estimation never changes the images sent to the model.

A context limit does not bound process memory. Current durable recovery reconstructs the Run’s original saved history range to verify context consistency, even after later compaction. Configure compaction before a Thread becomes large; enabling it later can still require a large initial read.

The default ContextCompactor.layer first prunes older application tool results, then makes a metered summary call if needed. It keeps protected instructions/input and complete tool pairs. Summary work consumes the Run’s budget. Durable compaction boundaries survive restart.

Install ContextCompactor.layerWithModel for a separate summarizer, or layerRollover to start fresh windows without a summary model call. Rollover retains original instructions and input plus bounded continuation notes. It preserves the Run’s cumulative charges and unresolved work; it does not start a new Run. Provider overflow permits one bounded compaction retry when a context allowance is configured; it is not unlimited retry.

Add ContextTools.toolkit and its handler Layer when the model should request new_context, inspect remaining capacity, or search/read retained context. new_context must be called alone. Supply ThreadContextHistory for durable history lookup. Searches use literal text, so short identifiers or exact phrases work better than a compound search expression.

MemoryNotes can bind read/write tools to one host-selected document for working notes across windows. The host owns its address, visibility, and storage. Notes and historical excerpts remain untrusted reference material.

RunContextPreparation transforms the source prompt before compaction. Its transientContext loader then adds current references for each Turn. These references consume context capacity but stay outside canonical history and summary coverage. A same-Turn overflow retry reuses the loaded snapshot.

Use the Agent’s inputPrompt to choose model-visible input. Preparation must preserve protected input and already-compacted prefixes; it cannot safely rewrite those boundaries. For dynamic model routing, return modelCall with capacity and output-reserve information matching the selected model. See model selection.

Memory.recall combines host-selected readers under item, byte, token, and time limits. Your application owns source selection and access checks. Keep recalled text in transient user context, never trusted instructions. Optional semantic retrieval ranks candidates, then revalidates them against current sources; an index match grants no access.

Correction and withdrawal affect future authoritative recall checks. They do not erase past outputs, captured views, or backups. The application owns background extraction, acceptance, and sharing policy.

toolResultBounds limits the encoded result retained in history and sent to the model (default 50 KiB). Oversized results become an explicit preview with the original byte count.

For follow-up reads, merge ToolResults.toolkit into the Agent’s toolkit and provide ToolResults.layer. Install a host store with Layer.succeed(ToolResultStorage, store) from @yielded/agent/tool-result-storage on the runtime Layer. The engine saves oversized outputs before returning a preview with fullResultId; read_tool_result retrieves up to 4,000 UTF-16 characters per call and returns nextOffset for continuation. Lower maxChars when using a smaller result budget.

Cloudflare hosts can use R2ToolResultStorage.layer(bucket, threadId) from @yielded/agent-platform-cloudflare/r2-tool-result-storage. Bind it to the authorized Thread; the model cannot choose another Thread. The sandbox example shows the runtime wiring. Other hosts implement the same Effect storage contract.

Saving has an 8 MiB ceiling and a five-second deadline. An expected save failure keeps the preview with fullResultError and no reference; it does not rerun the original tool. Without a store, omitted text is discarded. Hosts own private storage, retention, quotas, and orphan cleanup. Saved objects are separate from Thread exports and must remain available for as long as their references are needed. An expired or missing object returns a typed read failure.

See budgets for defaults and the context-continuity example for runnable rollover and recovery scenarios.