Context Engineering

Context assembly and authoritative task state

Context is the information made available to a model invocation. The context window is its bounded sequence capacity, with input and output accounting determined by the interface. Context engineering selects and maintains that information across calls; it includes instructions, conversation history, tools and external material. The context-engineering report describes this broader scope. [1]

Assembly starts with the immediate decision, gathers candidate information, checks eligibility, selects a useful representation and constructs the request. Application instructions, the user request, task records, observations and tool definitions have different roles. Keeping their serialization inspectable makes those choices testable instead of burying them inside a framework. [2]

The resulting input is a derived view: a representation built from underlying records. The materialized-view pattern supplies the analogy; a database materialized view is not required. [3]

A constructed incident investigation provides a running example. The assistant must investigate a service alarm without changing production. Its task record retains the target revision, completed checks, unresolved deployment verification and an external check still running. An operator has already requested deployment; the assistant's verification read timed out. Neither the request nor the timeout establishes the deployment outcome. [4][5]

Keep three things separate: authoritative application records, the history of observations and actions, and the current model input. A durable session log can retain material omitted from a call and supply it again later. A model proposal remains a proposal until the application accepts it through the appropriate state-changing process. [6][7]

An output contract specifies the expected response structure, including how missing information is represented. Structural acceptance does not establish factual correctness; Structured Outputs and Tool Calling develops that distinction. Persistent agent memory retains information for reuse across runs, but only selected material enters a particular call. Its storage and writing policies belong in Agent Memory. [7][8]

Instruction authority and examined data

A system instruction is application-supplied behavioral guidance carried through the interface's designated instruction mechanism. Roles and precedence are interface-specific. OpenAI's dated authority specification treats roles as structured metadata: a document declaring itself a system message does not become one. Conversation formatting explains the representation boundary. [9]

Prompt injection attempts to make untrusted content redirect behavior across an intended instruction boundary. In the incident, a retrieved note might recommend bypassing approval. That recommendation is material to inspect, not authority to change production. AI Security explains the threat; assembly must preserve the distinction between the application's constraint and the note's content. [10]

Separate instruction-bearing fields from examined material, and retain attribution when that material is rewritten. A summary can report that an incident note recommends bypassing approval without adopting the recommendation. Delimiters and encoding help frame content, but encoding alone does not establish a reliable instruction boundary. [11][12]

Permissions must remain enforced by application code for the specific operation and resource. Neither a retrieved instruction nor a correctly formatted model response grants access. This also applies to selecting private records for disclosure to a model, as explained in Authorization at access and disclosure boundaries. [13]

Evidence records and preserved meaning

Provenance records origin and transformation history. A source snapshot and its summary are distinct information entities connected by a derivation, as formalized in W3C PROV. [14]

An observation envelope is a useful application record that keeps evidence attached to its subject and limits. It is not a universal API schema. GitHub check runs illustrate the necessary distinctions: repository and run identity, head_sha, status, conclusion and timestamps. An in-progress run without a conclusion differs from a completed run whose conclusion is neutral. [15]

Preserve labels that determine meaning. In an example incident table, 'p95 latency (ms), staging only: 420; production not measured' cannot become 'production latency: 420.' The heading, unit, scope and negation travel with the value. Context-preserving document handoffs explain this boundary. Source attribution supports inspection; extraction still needs checking against the source. [17]

Failed, partial and unavailable observations need explicit status. A response can arrive without establishing success, and a timeout can leave an external outcome unknown. Preserve those limits alongside the operation and target. Results and justified completion claims covers the execution boundary; lineage and audit evidence covers its record. [5]

Selection for the next decision

Relevance means bearing on the question. Sufficiency means containing enough information to support an answer. Relevant records can omit a necessary connection; even sufficient context can contain false assertions. Required-fact coverage therefore deserves its own check. [18]

Retrieval finds external candidate material; Search and Retrieval explains how. Assembly decides which candidates join application state in the request. Nearest-neighbor search can return records even when none answers the question. A ranking is an ordering among candidates, not evidence that any candidate satisfies the information requirement. [19]

A practical selection map separates mandatory constraints, decisive evidence and optional background. For the incident's current-status report, the read-only constraint must survive; the current revision needs its own check observation; deployment status needs independent evidence. Old successful checks may explain history, but another copy adds no coverage of the missing deployment observation. [20]

Assess permission separately from usefulness before accessing a record and before sending it to the model. An application credential may reach information the requester cannot use. Current eligibility must come from trusted application state, not a model-supplied identity or a source's claim that sharing is allowed. [13]

Remove repetition without merging distinct revisions or disagreements. Two reports can share almost all their wording while differing in a date, number or negation that changes the decision. Retain identity and validity separately from duplicate-cluster membership; revision-aware deduplication explains why textual overlap is insufficient. [21]

Missing decisive evidence calls for another authorized read when feasible, or a bounded answer that identifies the gap. Irrelevant evidence calls for filtering. These are different repairs: increasing the number of returned records can add noise without supplying the missing fact. Selection should improve coverage and marginal usefulness, not merely increase volume. [20]

Complete-request budgets and overflow choices

Tokens are the model's sequence units, rather than a fixed number of words or characters. Budget the assembled request: instructions, history, evidence, tool definitions, output contracts and formatting. Complete-request token counts explains why independently estimated fragments do not establish the final count. [22]

Fit is a capacity condition, not a quality score. An application can choose a smaller operating budget, but no universal fraction is established. Unused capacity need not be filled with background that contributes nothing to the decision. [24]

Provider counting may be an estimate. Claude's documented endpoint accepts structured inputs, including tools and supported media, but actual usage can differ slightly and include provider-added tokens. Count for the intended model, preserve headroom for uncertainty and recount after reconstructing the request. An input estimate alone does not budget a complete multi-step interaction. [25]

Cached prefixes still occupy the context window: caching changes processing cost, not capacity. A file pointer is also not necessarily a small model input. In the demonstrated Gemini upload workflow, the backend loads the referenced file into context even though the client does not retransmit its bytes. [26][27]

Repair overflow explicitly: remove redundant logs, retrieve a narrower passage, narrow the decision or compact selected history. Then serialize and count again. Preserve mandatory constraints; shrinking the answer reserve is acceptable only if the remaining allowance supports the required output. [22]

Arbitrary message deletion can also break the interface. Preserve required tool-call/result pairings when trimming history. A smaller request that loses the relationship between an operation and its result is not a valid repair. [28]

Evidence ordering and coherent interpretation

Required message structure constrains ordering before editorial preferences enter. A tool result must remain correlated with its call identifier, not merely its function name or completion position. Authorized dispatch and call identity explains that contract. Evidence inside permissible structures can then be arranged for the decision. [29]

Chronology makes changes and their sequence easy to follow. Question grouping makes the evidence for each requested conclusion easy to locate. Either can be appropriate; neither permits detaching a qualification or silently changing wording. The comparison is an organizational choice to test, not a demonstrated ranking of formats. [30]

Lost in the Middle names historical research in which moving relevant information changed performance on selected question-answering and retrieval tasks. Several tested models favored beginning or end positions, with task-specific exceptions. This establishes a reason to test position sensitivity, not a universal rule for present-day models. [31]

Move the same decisive passage while holding the total content fixed. Vary distractor volume and total length in separate comparisons. Context access and demonstrated behavior explains why a passage's inclusion does not establish its successful use, and why answer failure alone cannot reveal internal attention. [31]

Loading and refreshing changing information

Freshness concerns validity for the current question. Source time, observation time and summary-generation time describe different events. Rewriting an old observation today does not make its facts current. [32]

When the incident target changes from revision A to B, A's successful check remains evidence about A. It does not become a B result. The same failure appears with cached business records: a reported deployment wrote a new credit score successfully, but a downstream agent read an older cached score because invalidation failed. [33]

Just-in-time loading waits until a decision needs information before acquiring its details. Progressive disclosure initially exposes compact descriptions or references, then loads fuller material when needed. A skill containing query-language guidance illustrates this pattern: its description helps select it, while its body supplies the syntax needed before execution. [34][35]

Preloading avoids later discovery work but retains material that may never matter. Demand-driven loading keeps the working input focused while adding retrieval steps and possible selection failures. Repository guidance can likewise load by subsystem; that guidance still needs maintenance when the implementation changes. [36][37]

Refresh when the target or task changes, a new result arrives, or an observation exceeds its permitted validity period. HTTP caching provides a concrete model of age and revalidation, but cache freshness is a reuse contract rather than proof that a business fact remains true. The application must define validity for its own decision. [38]

Recent observations need not form a consistent snapshot. Under PostgreSQL Read Committed isolation, successive SELECT statements can see different committed states, even within one transaction. An assembler that reads the target revision, then reads its status after another transaction changes the target, must reconcile identities rather than assume recency makes the pair compatible. [39]

If refresh fails, preserve the last observation's age and mark current evidence unavailable. Do not replace a timestamp with the latest retrieval attempt. Changed permissions also require reevaluating eligibility. Action-time revalidation adds a separate requirement when the system proceeds from reporting to consequential action. [38][13]

Compaction and summary fidelity

Compaction replaces accumulated context with a smaller representation for subsequent calls. Summarization creates a shorter account; retrieval loads selected external content. Durable notes remain outside the window until supplied again. These operations can cooperate, but they are not interchangeable, and a shorter input is not automatically more useful. [40]

Omission drops material. Exact extraction retains selected original spans. Structured projection selects fields from task records. Abstractive summarization rewrites information in new language. Each reduces detail differently; rewriting also creates opportunities to change relationships. Simple truncation can precede model-based summarization, provided omissions are marked and needed source material remains recoverable. [41][42]

A task summary should retain active constraints, decisions with rationale, unresolved questions, relevant observations, pending work and evidence references. A chronological recap can spend its space describing activity while omitting the obligations that remain. Reconcile the summary against current records before continuing. [40]

A source reference is a retrieval handle, not the source's contents. Explicitly reading the referenced record restores selected evidence to the next input. [44]

Summary errors can alter entities, predicates, circumstances and connections between statements. Inspect negation, uncertainty, scope and revision identity, not just fluency. FRANK's factuality taxonomy supports this relationship-level inspection; it does not measure modern agent-compaction failure rates. [42]

Repeated rewriting can also lose a qualification: 'cache mismatch is suspected' can become 'cache mismatch' and then 'the cause was cache mismatch.' This is a possible failure, not an inevitable drift law. Validate important claims against original observations. Fresh sessions with structured handoffs and continuous compacted sessions remain alternatives to test on the actual task. [45]

Source authority must survive rewriting too. A summarized incident note remains externally supplied content; it cannot grant approval. Untrusted content across transformations and memory covers this risk. Preserve requested, attempted, observed and verified status using the distinctions in justified completion claims. [10]

Trigger compaction with enough headroom for the transformation. In Claude's documented compaction beta, a threshold initiates an additional sampling iteration, and subsequent processing drops content preceding the compaction block. Compaction usage is reported separately. Submitted history, effective input and stored history can therefore differ; the API behavior does not guarantee summary fidelity. [46]

Exclusion, invalidation and derivative rebuilding

Eviction removes material from a working input to save space. Invalidation marks an assertion as no longer usable under its previous interpretation. Supersession supplies a replacement. A history can preserve the old assertion and the evidence that changed it, rather than silently overwriting both. [47]

A correction must reach dependent representations. If an obsolete note contributed to both an excerpt and a summary, excluding the note alone leaves two routes for its conclusion to return. Regenerate a derivative from eligible current sources, exclude it pending review, or retain it explicitly as historical evidence when that use remains permitted. [47]

Source deletion and derivative repair are distinct even in documented products. Zep states that deleting an episode does not regenerate summaries of nodes shared with other episodes; information from the deleted episode can remain there. Deleting an episode that invalidated a fact also does not automatically reverse that invalidation. [48]

Recheck eligibility when saved material returns through retrieval or conversation reconstruction. A previously permitted copy cannot establish present permission. A revoked note and an unchecked derivative should remain out of future inputs until their owning systems establish an eligible replacement. [13]

Excluding content from future calls does not delete stored transcripts, checkpoints or cross-run memory, and cannot retract an already transmitted request. LangGraph distinguishes current-message removal from checkpoint deletion and separate store operations. Broader erasure belongs in Lifecycle fulfillment across derivatives; persistent storage lifecycle belongs in Agent Memory. [28]

Verification across successive calls

Inspect what each call could actually receive, including selection and transformation decisions. A complete stored transcript is insufficient when the provider processes a compacted view. Check the boundary between submitted history and effective input, and distinguish known content from provider-controlled or unobserved processing. [46]

Long-session tests should start from retained history and evaluate a continuation. Sally-Ann DeLucia describes loading ten prior turns and testing the next, making late failures reproducible. The number is an example, not a coverage threshold. Include continuations that depend on old constraints, changed targets and information removed from the active view. [50]

Confirm that the policy activated. A short run that never compacts provides no evidence about compaction fidelity. Compare against untouched history where it fits, holding model, prompt, tools and tasks fixed. Initial compaction experiments reported that untouched history could outperform defaults, but their small trial counts did not establish a universal ranking. [51][52]

A context ablation changes one assembly choice while preserving other conditions: selection, ordering, compression or removal. Reset initial state and memory between independent trials, equalize budgets and repeat runs. Report task counts and uncertainty. Controlled offline comparisons provides the broader method. [53]

Choose settings on development cases, then evaluate the frozen policy on held-out tasks. Repeatedly modifying settings after inspecting the final test converts that test into development data. A policy that preserves this incident's distinctions still needs evidence that it handles different investigations. [54]

Capture only what the investigation requires. OpenTelemetry's GenAI conventions advise against recording full instructions and payloads by default; protected external references are an alternative. Recorded messages may also be truncated while retaining valid structure. Mark capture coverage explicitly: a valid trace object need not contain the complete request. Capture policy and execution provenance develops these controls. [55]

Open questions

1. 01

   Decision-dependent preservation remains difficult because a detail's importance may emerge only later. Progress would be a compaction policy that preserves required facts on held-out continuations while reducing context, including cases where an initially peripheral constraint becomes decisive. [40][54]

2. 02

   Ordering policies must separate position effects from changing content and task difficulty. A useful advance would demonstrate repeatable gains when identical evidence is rearranged across varied tasks and lengths, with distractor volume controlled independently. A single successful placement does not establish a transferable rule. [31]

3. 03

   Correction and revocation propagation remain hard when many derivatives share sources. Progress would be an inspectable dependency inventory that prevents a disallowed assertion from reappearing through summaries or saved history, while preserving eligible historical evidence. Deleting one source is insufficient when retained summaries are not rebuilt. [48]

4. 04

   Repeated compaction lacks a general fidelity guarantee. Its practical importance is preserving uncertainty and unfinished obligations across long work; the difficulty is distinguishing compression effects from evolving tasks and model variation. Progress would compare successive summaries against original records over repeated controlled continuations, reporting specific semantic losses rather than fluency alone. [45][53]