Prompting and In-Context Learning

Changing inputs, keeping weights fixed

A prompt supplies a language model with the task and information it should use to respond. The model predicts text using numerical values learned during training, called parameters or weights. Using those learned values to produce an output is inference; changing the prompt changes the input to that process, not the weights. Parameters and modeling assumptions explains their role. [1]

The model processes text as tokens, encoded units that can represent words, fragments or punctuation; Tokenization explains that representation. Predictions depend on the supplied input, a relationship called conditioning. Different instructions or examples can therefore guide different responses through the same weights. The computation produces temporary state, including any cached intermediate results, but that state is separate from the learned parameters. Following a convention in one request does not by itself make that convention available in a later request. [1][2]

This separation makes prompting useful: behavior can be adapted without a training run. This chapter concerns the instructions and demonstrations that guide a call. Context Engineering covers the broader application responsibility of selecting and maintaining everything supplied across calls, including history, external information and tools. [3]

Specify the intended task

An instruction describes an operation and expectations for its response. Specify the input, relevant constraints, intended use and treatment of uncertainty. Output style matters when it serves that use, but it cannot resolve an undefined decision rule. For example, requiring a short answer says little about which category the answer should name. Anthropic's prompting guidance recommends explicit requirements and explanations of their purpose. [4]

Consider a constructed specification for classifying messages. The purpose is to identify whether the sender requests an action, rather than merely detecting question marks or imperative verbs. Each clause below removes a different ambiguity. [5]

A prompt template keeps reusable instructions separate from request-specific variables, such as the message to classify. For a repeated API task, revise that reusable template when a failure exposes a missing rule. Correcting one answer later in a chat does not improve the independently invoked template. [6][7]

Before blaming the model, check that the requirements agree and the necessary facts are present. Demanding exactly one bullet and several bullets is contradictory. In the Prompt Doctor workshop, an apparent scheduling failure became acceptable uncertainty once the participants established that service availability had deliberately been omitted. [8][9]

Chat APIs label contributions with roles such as user and assistant. A model-specific chat template serializes these messages into the sequence the model expects; incompatible markers can impair behavior. Chat templates explains that interface. Within it, a format instruction remains a request. Schema-constrained generation adds structural enforcement, but valid structure can still contain wrong values. Structured Outputs and Tool Calling develops that distinction. [10][11]

Learning from examples in context

A demonstration pairs an input with its desired response inside the prompt. Zero-shot supplies none; one-shot supplies one; few-shot supplies several. These counts concern the request, not training exposure. Demonstrations show how to respond, whereas background facts supply information to use. Training examples participate in an operation that updates parameters; prompt demonstrations condition inference with fixed weights. [12][13]

In-context learning describes adapting behavior from information or patterns in the current input without a parameter update. The practical objective is generalization: useful behavior on inputs not shown in the demonstrations. Fit, select, and assess explains the broader distinction between fitting examples and succeeding beyond them. [12]

Demonstrations can communicate response conventions that are awkward to describe fully. A coding example can show the required component syntax and styling convention; paired writing briefs and finished pieces can show a client's tone. In both cases, the example supplies a pattern to apply to a new input. Whether the model follows it still needs testing. [14][15]

For the classification illustration below, replace familiar category names with dax for requests and wug for reports. Unfamiliar labels make the supplied association matter. Research using unrelated label names found that some tested models followed these mappings more successfully than others. Correctly answering one new message would establish local success, not a particular internal learning algorithm. [16]

From continuation cues to instructions

The distinction between training a model and adapting through its inputs predates language-model prompting. In 2001, Hochreiter, Younger and Conwell investigated learning to learn, also called meta-learning: training a system to acquire a procedure for learning related tasks. Their recurrent network carried information from successive inputs and previous target values in its internal state while predicting changing numerical functions. Training shaped its ability to adapt; new examples supplied information for that adaptation. This provides a conceptual precedent involving numerical tasks rather than language instructions. [17]

GPT-2's 2019 experiments communicated tasks through continuation cues: a summary cue followed an article, while translation used sentence pairs and an unfinished pair. These formats directed completion, although generated summaries still confused details. [18]

GPT-3's 2020 study made inference-time demonstration counts an explicit comparison: task descriptions alone, one demonstration and several demonstrations, all without task-specific weight updates. This helped distinguish adaptation through a request from adaptation through training. [12]

A complementary problem was making models more responsive to direct instructions. Instruction tuning updates a pretrained model's weights using tasks expressed as instructions and examples of desired responses. FLAN, first reported in September 2021, studied this approach. Its evaluation withheld entire groups of task types from instruction tuning and compared the resulting model with its untuned counterpart. The tuned model performed better on those unseen task types, although their exclusion from instruction tuning did not establish their absence from pretraining. [19]

The BigScience T0 project, first reported in October 2021, trained on multiple tasks expressed through varied prompts. More prompt formulations per dataset improved median held-out performance and reduced prompt variability in its experiments. This was a training intervention; adding paraphrased instructions to one request is a different operation. [20]

Training for instruction following and supplying completion cues address different parts of the interaction. A base model—a pretrained model before instruction tuning—may continue a coding request with descriptive prose, while a function definition and docstring elicit code, as illustrated in Decoding Mistral AI's Large Language Models. The example shows how an unhelpful response can reflect the interaction format rather than missing coding capability. Training changes the model's responsiveness; instructions and demonstrations still specify what it should do in a particular call. Post-training and Alignment explains parameter adaptation. [21]

Select examples for useful coverage

Choose demonstrations for the distinction they communicate. First verify that their desired responses follow the task specification. Then consider relevance and coverage: which important cases the set represents. Repeated direct requests may leave polite questions unexplained. A contrasting example can clarify that boundary more directly than another near-duplicate. Negative examples are useful when their explanation identifies a misconception, rather than merely declaring an answer bad. [22][5]

When different requests need different examples, dynamic few-shot selection chooses demonstrations for each query from a larger collection. It can use explicit rules, such as matching a category, or search for example inputs similar in meaning to the query. Search and Retrieval explains how to find such candidates. The selection procedure must still choose a set that demonstrates the intended behavior together. [23]

KATE tested similarity selection for sentiment classification: deciding whether text is positive or negative. It prompted GPT-3 with three labeled sentences from SST-2, a sentiment dataset, and assessed predictions on IMDB movie reviews, keeping the generation setting fixed at temperature zero. Random selection averaged 87.95% accuracy, with a 2.74-point standard deviation across five runs. Similarity selection using an unchanged RoBERTa-large encoder, which represents text numerically for comparison, achieved 91.99%. Selection also determined example order, so the result compares selection procedures rather than isolating similarity alone. [24]

Selecting each example independently can nevertheless produce a redundant set. Coverage-based selection research found its strongest benefits on compositional semantic parsing—turning language into programs or queries that combine operations. The method did not consistently improve ordinary classification or numerical reasoning. Useful diversity depends on the task. [25]

Examples also compete for capacity. The context window bounds the tokenized sequence available to a request. When input and output share that allowance, adding demonstrations leaves less room for the query or response. Count the complete input, including instructions and template overhead, and reserve output space while respecting any separate input and output limits. For models that generate internal reasoning tokens, that reservation must cover them as well as the visible answer. Counting complete requests covers the accounting. Caching reuses processing of an unchanged input prefix; it can lower processing cost but does not free context capacity. [26][27]

More available space does not imply steadily improving results. Many-Shot In-Context Learning tested the original February 2024 Gemini 1.5 Pro using smaller demonstration sets nested inside larger ones and repeated selections. On 150 test articles from the XSum news-summary dataset, scores improved through roughly 50 examples and then declined. Using XSum demonstrations to summarize articles from another dataset, XLSum, generally improved with more examples. Scores used ROUGE-L, a measure of overlap with reference summaries; fabricated dates and times still occurred. These different curves give no universal shot count, and overlap scores do not establish factual accuracy. [28]

Order and formatting can change behavior

Prompt sensitivity is behavioral variation associated with changes in wording, examples, order or presentation. Some edits change the task. Others intend to preserve its meaning but change the sequence the model receives. These require different interpretations. [29]

A controlled order study evaluated all 24 permutations of four fixed, balanced SST-2 sentiment demonstrations across GPT-2 and GPT-3 sizes. Accuracy varied substantially, and effective orderings correlated weakly across models. Increasing example counts did not reliably remove the variation. These historical results establish order as a variable worth testing, not a preferred permutation for current models. [30]

Label bias is a preference for an answer label that need not follow the current input's meaning. Experiments found preferences for frequent demonstration labels, recently shown labels and familiar answer tokens. Balanced examples expose each class, but do not necessarily reproduce deployment prevalence or remove other biases. Choose the class mix against the intended workload and consequences of errors. [32][33]

Formatting can also change token boundaries, although tokenization alone does not explain every sensitivity result. Consistent separators help distinguish instructions, examples and query. The Prompt Doctor workshop's useful clarification was that clear separation mattered more than XML itself. Such markers do not enforce a security boundary: external text can still be interpreted as instructions. Context Engineering covers handling that untrusted input. [34][35]

Separate development from assessment

Define success before selecting a prompt. For the message classifier, distinguish a correct category from a correctly formatted response, and identify consequential mistakes such as overlooking a request embedded in a report. Establish a baseline, the reference configuration: a clear instruction-only prompt and, when one exists, the prompt already in use. Include conventional code when it is a meaningful alternative. [11]

Start by saving baseline responses on concrete cases and rerunning those cases after a change. The Prompt Doctor workshop demonstrates why this helps: improved formatting did not automatically resolve the remaining behavioral failures. This establishes a useful development practice, but the cases that guide revisions are not independent evidence about the selected result. [36]

Separate three uses of examples. The demonstration pool supplies material that may enter prompts. Development cases provide feedback for choosing instructions, examples and presentation. A held-out assessment evaluates the frozen choice without guiding its selection. The True Few-Shot Learning study showed why visible shot count understates development effort: selecting a prompt can require many more labeled examples than appear inside it. [37]

Leakage occurs when assessment information crosses that boundary. Putting a test answer into a demonstration is an obvious route; splitting near-duplicates or related messages across pools can also weaken the intended test. Adaptive overfitting occurs when repeated test feedback guides prompt selection. The weights can remain unchanged while the prompt becomes specialized to the repeatedly consulted cases. Keep assessment independent explains these routes.

Record the full comparison configuration: model version, surrounding input, chat template, demonstration identities and order, generation settings, attempt limits, output allowance and scoring procedure. Preserve relevant conversation turns, not just the instruction template. A workshop reproduction of a JSON reviewer needed upstream tests that had arrived through conversation history; changing that history or the model changed the experiment. [38][39]

Use shared cases and explicit budgets throughout development. Once the candidate is frozen, compare it with the baseline on held-out cases. If those results drive another revision, they have become development feedback; a fresh independent assessment is needed for the revised choice.

Interpret controlled comparisons

A paired comparison evaluates alternatives on the same cases, making it possible to see which answers changed from wrong to right and which regressed. An ablation removes or changes a component to investigate its contribution. Decide whether the experiment compares complete prompt packages or isolates a narrower change. Holding other conditions fixed makes that distinction interpretable. [40]

FormatSpread held demonstration identity and order fixed while varying separators, capitalization and option numbering. Across 53 tasks and the tested model and demonstration-count settings, the median gap between the best and worst of ten sampled formats was 7.5 accuracy percentage points. The study used Llama-2 and Falcon variants; its principal scoring method chose answers by their model-assigned probabilities. This measured sensitivity within the tested format choices. It does not predict a gain of 7.5 points from reformatting a current application's freely generated responses. [31]

For equally weighted cases, subtract regressions from improvements and divide by the number of assessed pairs to obtain the accuracy difference. Then inspect task slices, relevant subsets such as mixed messages or polite requests. A positive total can conceal unacceptable regressions on a consequential slice. Repeatable evaluations expose these tradeoffs more reliably than choosing whichever response looks most impressive. [42]

Record complete input length, output usage and measured latency separately from correctness and format compliance. Prespecify repeated trials when output variation matters, and retain the same attempt policy for alternatives. If several components changed, the result belongs to the bundle. It does not establish which component caused the gain. [40]

Finally, assess uncertainty in the difference, not only the two headline scores. A small apparent gain can remain inconclusive, particularly on rare but important cases. The independent comparison supports the frozen candidate under the tested conditions; it does not establish superiority on other tasks or models.

Evaluate requests for intermediate steps

Chain-of-thought prompting asks for, or demonstrates, intermediate reasoning before an answer. The 2022 demonstrated method added worked steps to input-answer examples and reported gains on tested arithmetic, commonsense and symbolic tasks. It changed both the information supplied and the generated work, so those gains do not establish equal-cost superiority. [43]

A zero-shot request asks for intermediate steps without worked demonstrations. The published 2022 zero-shot protocol used two stages: generate reasoning, then append an answer-extraction instruction and generate the final answer. Reproducing that result requires retaining the additional stage; it was not simply a one-call wording change. [44]

Compare such a protocol with direct answering under explicit call and output limits, and measure actual usage and latency. Score the answer independently of the explanation's fluency. Experiments on GPT-3.5 and Claude 1.0 found that answer suggestions and demonstration positions influenced predictions while explanations often omitted those influences or rationalized mistakes. Plausible explanatory text therefore need not faithfully account for what changed the answer. [45]

Deliberation is additional work toward a solution; verification checks a proposed result against independent criteria. Reasoning and Test-Time Compute develops those mechanisms. Here, requesting intermediate steps is another prompt choice to evaluate for the particular model and task.

Bound the transfer claim

Finite demonstrations can support several interpretations. A model may recognize a familiar task, copy its response conventions or apply a newly demonstrated mapping. In classification and multiple-choice experiments, randomized demonstration labels often retained much of the benefit; label vocabulary, input distribution and formatting each contributed. Thus, improved output alone need not show that the intended input-label rule was learned. [41]

Unfamiliar-label experiments probe a different ability: following associations supplied locally. Some models succeeded, while others relied more on familiar label meanings. Instruction-tuned PaLM variants improved at unrelated labels but resisted demonstrations reversing familiar meanings. These contrasting results show why a single account of what demonstrations teach is insufficient. [16]

Compositional generalization means applying operations in combinations absent from demonstrations. Coverage-based selection helped particular compositional parsing splits more than ordinary classification. That is a narrower achievement than general transfer. Likewise, order preferences differed across tested models, and many-shot summarization gains changed with the evaluation dataset. [25][30][28]

Design the test around the condition that changes. Distribution shift means that input–outcome conditions differ from those used to establish performance. Merely calling every assessment input new does not specify how demanding a transfer claim is.

For dynamic selection, freeze the selector and example pool while allowing the query to determine the selected examples. The claim then concerns transfer of a procedure, not identical demonstration text. If target results guide revisions, describe the outcome as target adaptation and assess the revised procedure independently. [23]

Choose the next intervention

When prompt revisions stop helping, several explanations remain possible. The task may be unclear, demonstrations may miss a boundary, necessary information may be absent, or the model may struggle with the required operation. A plateau does not distinguish these causes. Investigate a specific hypothesis instead of accumulating more instructions.

These interventions change different parts of the system. Context Engineering changes which information reaches an invocation. Post-training and Alignment changes learned parameters using training data and objectives. Prompting still specifies the task for the adapted model, so the two approaches can coexist. Neither a new training procedure nor a new source of context guarantees improvement. [3][46]

Apply the acceptance criteria established before selection: task quality, consequential slices, output requirements and resource limits. Retain a prompt that meets them with adequate independent evidence. Revise a demonstrated weakness when a plausible input change addresses it. Narrow supported use when evidence covers only part of the workload, or investigate another intervention when the remaining failures warrant it. Every revised choice needs evidence appropriate to its new claim.

Open questions

1. 01

   Predicting which demonstrations add useful coverage remains difficult: similarity can retrieve redundancy, while coverage depends on the chosen representation. Progress would predict marginal usefulness across tasks before extensive prompt search. [25]

2. 02

   Prompt portability needs stronger evidence across model versions and languages. Presentation preferences can change across models; useful progress would establish performance ranges under declared changes without target-specific retuning. [30]

3. 03

   Demonstration-following still leaves ambiguity about what was acquired. Distinguishing format imitation, familiar-task recognition and new mapping use matters for transfer; progress requires interventions that separate those explanations. [41]

4. 04

   Efficient prompt selection must preserve assessment independence despite limited labeled data. Progress would reduce search and labeling costs while maintaining reliable estimates on cases excluded from selection. [37][47]