AI Engineer Summit 2023
Writing Principles for Task-Tuned Prompt Engineering
Read the talk
Writing Principles for Task-Tuned Prompt Engineering
Karina Nguyen works through recommendation prompts, independent question decomposition, long-document evaluations, and production features to show how task-specific writing shapes model behavior.
From a talk by Karina Nguyen
Before you start: Basic familiarity with language-model prompts is helpful; embeddings, decomposition, and the evaluation procedures are explained as they appear.
How do you turn a goal into a working prompt?
You know what you want a language model to do. How do you explain it so the model actually does it well? That gap between intent and performance is the opening problem of Karina Nguyen’s workshop, drawing on her work reducing hallucinations and improving self-correction in Claude 2.
A language model estimates the probability of the next word given the preceding text. A prompt changes that context: it supplies task-specific language, directs attention toward relevant information, and provides examples of the desired continuation. This is a way to use inference-time computation without retraining the model. But the difficulty can begin at different levels:
- A clear goal, an unclear method: You know the desired outcome but do not know how to get the model’s best performance.
- A vague goal: You have an approximate idea but cannot explain it precisely enough.
- An undefined goal: You have not yet decided what a successful answer should be.
These are different problems. More elaborate instructions cannot fully resolve a task whose objective is still undefined.
When the task is hard to articulate, start with examples. Make them diverse enough to include edge cases, then explain the request in simple language. Nguyen describes repeated iteration and collaborative prompt writing as a substantial part of her research work: the prompt improves as the people writing it become clearer about the task.
Her comparison is to creative-writing exercises she encountered at Berkeley. Prompting requires trying different ways of expressing an idea, rather than assuming the first formulation should work. In connection with her essay Cultures of writing, she gives this practice an experimental structure: form a hypothesis, test an assumption, and revise. Can the model self-correct an answer? Try a prompt that makes the behavior observable. Each attempt reveals something about the model’s strengths and weaknesses on that particular task.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Give the model enough guidance to act
A useful prompt communicates the objective and supplies just enough constraints to guide the response. Nguyen’s Claude-specific advice is to explain the request as carefully as you would to another person. That can require elaboration, but elaboration should clarify the task rather than add unrelated material.
| Principle | What to put into practice |
|---|---|
| Clarity | Simple language and unambiguous instructions |
| Conciseness | Only information the task needs |
| Coherence | Context first, a clear task at the end |
| Consistency | Stable terminology and formatting |
| Direction | Genre, length, style, and response constraints |
| Grounding | Examples, sources, and supporting quotations |
| Diversity | Examples that include distinct cases and edge cases |
The principles reinforce each other. Consistent XML tags distinguish the parts of a prompt; explicit output requirements tell the model what to produce; quotations from documents or search results give an answer something concrete to rest on.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From a clothing query to candidate items
Nguyen’s first application is Interalia, a clothing-search project built from images and items collected across brands. A user can search for a James Bond girl look or a futuristic ethereal outfit, retrieve clothing with that aesthetic, and follow a result to the shop. These requests describe associations and style rather than catalog fields such as material or sleeve length.
The retrieval mechanism uses CLIP, OpenAI’s Contrastive Language-Image Pretraining model. Encode the clothing images and the text query into a shared embedding space, then rank database items by cosine similarity:
Here, q is the query embedding and i is an item’s image embedding. The shared representation makes text-to-image retrieval possible. As the audience discussion clarifies, this is a particular multimodal connection between text and images; audio requires other approaches.
Retrieval supplies candidates. The next question is whether a candidate actually satisfies the request. Consider a dress in Emma Chamberlain’s style, a blazer like one from The Great Gatsby, or a futuristic outfit for the Met Gala. Nguyen proposes giving Claude the user’s query and labels describing each image. Those labels can come from the original product source or from a multimodal model. Claude then judges relevance: should this particular item be recommended to this particular user?
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Make the relevance decision explicit
The simplest curation prompt asks for a binary judgment. Its structure separates the user’s intent from the item description and gives the response a predictable location:
I need you to decide whether the item is relevant to the user query.
<user_query>
{user_query}
</user_query>
<item_description>
{item_description}
</item_description>
Should this item be recommended to the user based on the query?
Answer yes or no. Write the answer inside <answer> tags.
The direct instruction establishes the task; the descriptive tags establish the boundaries of its inputs. Nguyen emphasizes using the same tag names consistently.
An audience question probes why XML works well for Claude. Nguyen says XML was the first format used in fine-tuning, with Markdown and JSON support following customer needs. Asked whether the preference came from fine-tuning or the underlying training data, she tentatively attributes it to both. The advice is grounded in her experience with the Claude models of the workshop, rather than a format-independent rule about language models.
The audience also notices missing closing tags in the slide, which Nguyen acknowledges as a mistake. Balanced tags matter when the application needs to extract the response. A further instruction—put nothing outside the answer tags—aims to suppress the conversational preamble that otherwise complicates parsing. The examples shown through Claude.ai produce yes and no decisions, but Nguyen treats them as an imperfect zero-shot starting point to improve through iteration.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Add criteria before adding a score
The next iteration asks Claude to consider the item before making its decision, writing that intermediate critique inside thoughts tags. More importantly, it specifies what relevance means. Does the item match the requested attributes? Does it suit the season or weather? A winter coat should not be recommended for a summer request. These criteria turn an underspecified judgment into a set of concrete checks.
Nguyen then replaces the binary answer with a recommendation score from 1, least recommended, to 10, highly recommended, returned inside score tags. The two demonstrations show the intended distinction:
| User request | Candidate | Demonstrated score |
|---|---|---|
| James Bond blazer | Blazer judged relevant | 9/10 |
| Dress in the style of The Great Gatsby | Braided cord cropped waistcoat | 2/10 |
The waistcoat fails the request’s garment and style cues. The critique supplies the explanation for that low score; the score alone would hide which requirement the item missed.
Two audience questions expose different limits. First, why can a well-structured prompt written in incorrect English still work? Nguyen suggests intent inference and transfer across languages, while another participant suggests that a small syntax error leaves the text close enough to the intended pattern. Neither offers a firm causal explanation.
Second, what does the distribution of recommendation scores look like? Are the scores biased, or normally distributed? Nguyen connects the question to ongoing societal-impact research on bias in news summaries, but says the answer depends on the task. She had not tested this recommender’s score distribution; the prompting example was prepared the previous day. A numeric output makes ranking possible, but this demonstration does not establish how to interpret its scale across a catalog.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Separate the subquestions, then recombine the answers
The workshop’s next resource is Claude’s then-available 100K-token context window, large enough to hold The Great Gatsby. Long context can contain multiple documents for summarization or retrieval, but it can also hold a large collection of few-shot examples. Both approaches spend more computation at inference time to make the task easier to specify.
More generated reasoning does not necessarily provide a faithful explanation of an answer. In Question Decomposition Improves the Faithfulness of Model-Generated Reasoning, faithfulness concerns whether stated reasoning reflects the process that produces the final answer. A plausible chain of thought may not actually determine that answer. Nguyen reports that decomposition-based methods sometimes approach chain-of-thought performance on question answering while improving faithfulness.
The concrete question is: could Scooby-Doo fit in a kangaroo pouch? The choices are A, yes, and B, no. A conventional chain-of-thought prompt asks the model to think step by step, then select the most likely answer; the demonstration selects B. Decomposition changes how the supporting information is obtained:
- Split the question into atomic, self-contained subquestions.
- Answer each in an independent context, so a preceding answer does not steer the next one.
- Recompose the evidence by putting the subquestions and their answers into a shared context.
- Select the final answer from that assembled information.
The example subquestions ask what kind of animal Scooby-Doo is and how large an average kangaroo pouch is. After recomposition, the model again selects B. The distinction is the separation between intermediate contexts, not merely the presence of a numbered explanation.
When the audience asks for the actual decomposition instruction, Nguyen turns to a legal multiple-choice question: which argument most persuasively establishes someone’s liability to a creditor under an agreement? A large few-shot prefix precedes the problem. The operative instruction asks for concise subquestions that contain all the information needed to answer them, without decomposing more than necessary:
I am going to give you a question.
Decompose it into a series of subquestions.
Each subquestion must be self-contained and include all information
necessary to answer it. Do not decompose more than necessary.
Be concise. Put each subquestion in a numbered tag:
<subquestion_1>...</subquestion_1>
<subquestion_2>...</subquestion_2>
<question>
{legal_question_and_answer_choices}
</question>
Self-containment is what makes a subquestion usable outside the original conversation.
The model generates the subquestions, which are answered separately before all the answers are supplied for a final selection. One asks about consideration in contract law; the demonstrated final choice is C. Different models can answer different subquestions. The workflow therefore separates decomposition, answering, and recomposition into distinct operations rather than requiring one model to carry the entire task in one context.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Build an evaluation that requires reading the document
A long-context evaluation must distinguish reading from recall of training data. If a model already knows the answer, a correct response says little about whether it used the supplied document. Nguyen describes an Anthropic experiment using government departmental meeting transcripts dated July 13 of the workshop’s year, chosen because they came after Claude’s training-data cutoff. The target was specific: correctly retrieve a piece of information from a long document.
The dataset construction uses Claude itself:
- Split the source document into sections.
- Generate five factual multiple-choice questions for each section.
- Give each question one correct answer and three incorrect alternatives.
- Reassemble randomized sets of sections into long documents.
- Ask the questions against those documents and measure correct answers.
This creates an evaluation whose questions and source passages can be rearranged while preserving their relationship.
The prompting comparisons include direct questions, two fixed examples of unrelated general knowledge, and two or five correctly answered examples drawn from the document context. The experiment also varies where the answer-bearing section appears: beginning, middle, or end. Nguyen describes testing 70K- and 95K-token documents. The companion article, Prompt engineering for Claude’s long context window, provides the detailed prompts and methodology.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Put the question after the document
The metric is how often the model answers a multiple-choice question correctly. In this long-document Q&A experiment, Nguyen reports that placing the question at the end of the prompt performed better than placing it at the beginning. She also reports that extracting relevant quotations into a scratchpad before answering improved accuracy at a small latency cost. This is a second position variable: the question’s location in the prompt is distinct from the answer-bearing passage’s location inside the document.
The tests include Claude 2 and Claude Instant 1.2. Nguyen describes larger gains from prompting for Instant; the companion study evaluates Instant extensively and compares Claude 2 on the baseline and the strongest Instant strategy. That setup supports a task-specific prompting comparison, not a general claim that Instant is the more capable model.
A scratchpad here means a text region, such as thoughts tags, produced before the final answer. For document Q&A, it can collect the relevant quotations before the model chooses an answer. Nguyen’s practical recommendation is to put the instructions at the end. An audience member asks whether end placement mattered more than the other tested optimizations, and she agrees.
Asked why end placement helps, Nguyen offers distance and stronger attention to the end as hypotheses. The discussion also raises the possibility of models retaining information at the beginning and end while losing information in the middle, but she says she has not read the referenced paper carefully. She explicitly limits the finding to long-document Q&A; to her knowledge, the team had not tested the same placement claim on other tasks.
A separate question compares prose plus XML with Markdown and all-XML prompts. Nguyen says XML has been more reliable in her experience than JSONL or Markdown, describing formatting accuracy as almost 100%. She does not supply a defined task, denominator, or evaluation procedure for that estimate, so it should be read as practical experience with these models rather than as another result from the document-recall experiment.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Name clusters and teach the boundaries between categories
Another task is turning groups of texts into readable categories. Nguyen describes representing texts with UMAP and applying K-means clustering, correcting an initial reference to KNN. For each cluster, aggregate its statements or claims and ask Claude for a category name. The cached labels shown were generated with Claude 1.3; she expects Claude 2 to do better, rather than presenting a measured comparison.
A single generated label is only one sample. Self-consistency introduces independent samples and a way to compare them:
- Majority vote: Generate
Nindependent answers and select the most common. Nguyen describes this as especially useful for quantitative questions, such as math problems, where answers are straightforward to compare. - Pairwise consistency: Generate two answers and ask another model whether they agree. Retain consistent answers and deselect inconsistent ones.
Agreement supplies additional confidence, although it does not by itself establish that the shared answer is correct.
If the model misses category nuance, improve the instruction’s distinctions rather than only requesting another sample:
- Contrast good and bad categories: Identify a label that is too narrow or too general and explain why it fails.
- Vary the setting: Use examples from different contexts so one surface pattern does not define the category.
- Explain difficult concepts: Use an analogy, metaphor, or decomposition when a definition alone is insufficient.
- Correct misconceptions: For a concept such as false presupposition, show a mistaken interpretation and explain the error.
An audience clarification makes the division of labor explicit: the clustering algorithm creates the groups; the language model names them.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Respect the conversation format and allow missing answers
Nguyen’s API examples use the historical Claude text-completion protocol: \n\nHuman: introduces a human turn, and \n\nAssistant: introduces an assistant turn. These are formatting instructions for the 2023 demonstration. The current Messages API instead represents turns with role and content fields, and assistant-prefill support depends on the model.
Within that historical protocol, Nguyen suggests prefilling an assistant response with an acknowledgment or asking Claude to repeat the instructions. The aim is to establish the intended behavior before the substantive task. Her next recommendation is equally practical: give the model permission to report insufficient information. A prompt that requires an answer regardless of evidence gives the model no explicit route to abstain.
For quotation extraction, define both the successful output and the empty result:
<document>
{document}
</document>
<question>
{question}
</question>
Find direct quotations from the document that are relevant to the question.
If there are no relevant quotations, say:
I don't find any relevant quotes.
Do not fabricate quotations.
The no-match response matters because a request to find quotations can otherwise encourage fabricated evidence. Few-shot examples need similar care: use cases resembling the real task, include enough diversity to avoid teaching incidental details, and balance answer types instead of repeatedly making option A correct.
The formatting Q&A uncovers a more specific mistake. Nguyen first suspects missing double newlines, then identifies Human: and Assistant: inside XML-wrapped example content. In the old protocol, those strings mark conversation boundaries; they should not also serve as casual speaker labels inside embedded text. Use labels such as user and AI, or H and A, for that content. Actual conversation turns should alternate human, assistant, human, assistant—not human, human, assistant, assistant.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Prompting becomes part of product development
Nguyen expects prompting to remain useful as the tasks become more complicated and nuanced. Synthetic data generation is one direction: models can produce diverse datasets and help write evaluations. Another is reinforcement learning from AI feedback, where prompted reflection and revision contribute to a process that can scale beyond reliance on human feedback alone.
The product applications can be small. Claude’s automatically generated conversation titles are one example of a prompt supporting an everyday interface feature. Nguyen also proposes embedding users’ conversations to suggest personalized topics. Alongside new capabilities comes an optimization problem: find a prompt that preserves task accuracy while using fewer tokens. She points readers toward the Anthropic Claude Cookbooks, prompt-design documentation, and research-paper appendices for practical prompts and retrieval or search examples.
The final questions test how far these ideas transfer. Nguyen’s experience is primarily with daily Claude use; she has less experience with GPT models and has not examined their API documentation closely. She does not offer a cross-provider evaluation of the workshop’s techniques.
An audience member raises Promptbreeder, which uses language models to improve prompts in light of outcomes. Nguyen identifies automated optimization as a promising direction and recalls the related work Large Language Models as Optimizers. Self-correction remains a motivating target: despite being easy to request, it is a behavior she says models still struggle to perform reliably. The useful search is for prompts that improve a specific weak capability.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Evaluation depends on what the feature must get right
How should a team evaluate response quality automatically? Nguyen’s answer is task-dependent. Some outputs need qualitative inspection. For refusal behavior, a model can categorize generated responses, including whether a refusal fits the context, and those categories can support a refusal-rate calculation. For hallucinations, she identifies direct human inspection as part of the work. Automation begins with deciding what kind of defect the evaluator is trying to detect.
Another question proposes function calls with enumerated types: predefined classes or user intents could constrain classification outputs. Nguyen says she has not used OpenAI or other function-calling systems extensively, so the discussion supplies no result about whether those constraints improve classification accuracy.
Conversation titles provide a more concrete production example. Nguyen prompted Claude to act as an editor with editorial taste, then included previous titles in the context when generating a new one. The prior titles supply examples of the user’s style, giving consistency a visible reference. Asked for precision and recall in detecting title defects, she does not share figures, but confirms that the feature is used in production.
The implementation also avoids generating a title when it is unnecessary. For very short prompts, Nguyen says Claude.ai can take the first words instead of invoking a language model. The interface examples include an introduction and a request for book recommendations. The decision is about the feature’s needs: a short input may already contain a serviceable title.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Model tradeoffs and the behavior red teaming tests
The closing discussion turns to Claude Instant, which an audience member credits Nguyen with training. Claude Instant 1.2 was announced on August 9, 2023. Nguyen describes Claude 2 as larger and more capable overall, while Instant is cheaper and faster. She reports that Instant 1.2 improved over Instant 1.1 on math, code, and reasoning tasks, with other benchmarks remaining similar. Math and code were explicit training targets.
Nguyen also reports improved robustness to jailbreaks in automated red-team evaluations. When asked what she means by training in this discussion, she clarifies that she means fine-tuning. The distinction matters: the earlier prompting methods change the context at inference time, while fine-tuning changes the model through training.
The final question asks what red teaming actually tests. Nguyen gives an unsafe-request example: an attempt to obtain bomb-making instructions. The desired response is refusal without supplying additional unsafe information. Internal evaluations probe how robustly the model maintains that behavior under jailbreak attempts, and she directs readers to the Claude 2 model card for evaluation details. The displayed automated red-teaming chart labels lower values as better: success here means resisting attempts to elicit the unwanted response.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
Nguyen's companion essay on writing cultures and innovation.
OpenAI's image and text embedding model, with installation and similarity examples.
Research on answering subquestions in separate contexts to improve reasoning faithfulness.
The 2023 long-document recall experiment, including synthetic questions, quotation scratchpads and contextual examples.
Anthropic's evolving collection of practical Claude notebooks and recipes.
An evolutionary method that improves task prompts and the prompts used to mutate them.
OPRO generates candidate solutions using earlier candidates and their evaluated objective values.
The historical release announcement with math, coding and safety comparisons against Instant 1.1.
Anthropic's historical documentation of Claude 2 capabilities, alignment and safety evaluations.
Updates since the talk
Current Claude message formatting and model-dependent guidance for assistant prefilling.
Read the complete timestamped transcript
- 0:01
My name is Karina. Um, recently I've been working on Claude, which is, like, a large language model trained by Anthropic. And, um, most recently I was working on, like, reducing hallucinations, how to make Claude self-correct its answers, and many other features that went into a Claude 2 launch.
- 0:26
And so I'm gonna talk today about the writing principles for task-tuned prompt engineering and kind of sort of, like, help you if you want to use Claude API, help you to guide what's the best practices and tips that I found and we found most effective.
- 0:45
So first of all, um, I would like to talk about why prompting is hard. And to understand why prompting is hard, we should understand what prompting is in the first place.
- 0:56
So these models estimate the probability of each subsequent word given the preceding words. So in a way, a well-crafted prompt can increase the probability of generated desired and accurate, um, phrases.
- 1:12
Due to attention mechanisms in large language models, the models can focus on specific parts of the input text. And so effective prompts ensure that the attention is directed, uh, appropriately for desired outputs.
- 1:27
And so it's important to incorporate, like, task-specific keywords and context and examples within the prompt to activate the relevant portions of the model's internal knowledge. And lastly, like, you know, prompting leads to better results because without the need of, like, computational, like, other compute, you just, like...
- 1:49
W- without, like, model retraining. So, uh, you can, like, leverage inference time to test compute for this. And so, like, why prompting is hard. And I think I found based on, like, my conversations with customers and, like, developers, I think prompting is hard because of, like, three different reasons.
- 2:06
First, people know, humans know what they want, but they don't know how to get the best performance from the model, and I think that's what we're gonna focus on today.
- 2:17
The second reason is that they vaguely know what they want, but they don't know how to explain the best to the model, and so the model gets confused what the human wants from the task.
- 2:31
And the third reason is, like, they don't know what they want. So the humans don't know what they want, so it's pretty bad. [laughs] Um, and it's hard for the model to understand.
- 2:40
So basic strategies that you can, like, if you find yourself, like, you kind of vaguely know what the task is, just provide a bunch of examples, and the model will be good at inferring what you're trying to do just based on, like, examples.
- 2:57
And the examples may be, um, should be diverse and should en- encapsulate a bunch of, like, edge cases.
- 3:06
Try to explain as you would have to explain to a [REDACTED:age] or, like, very, you know, in a very simple terms. And I think what I found is that, like, as you have to, like, be able to, like, iterate a lot and, like, spend a lot of time just prompting.
- 3:24
And in a way, based on my experience, for example, as, like, research engineer, I spend the majority of pairing with people just collaborating on the prompts.
- 3:35
Um, in the past I was thinking, like, how my main experience with the word prompt was only in the creative writing classes. Uh, I graduated from Berkeley a- and I took some, like, creative writing classes, and we usually have, like, exercises, like prompting exercises, right?
- 3:52
And so we often forget that prompting language models is actually an act of creative writing. And I see people get annoyed, like, why their prompts just don't work. But in most cases, I think it just means that people lack some kind of, like, originality or creativity to think, uh, of, like, new novel ways how to make it
- 4:12
work. Um, I recently wrote a blog post about the cultures of writing, and one of the points that I'm making in the blog post is that prompting becomes like a form, a new form of writing for any research engineer and, like, scientist who engages in daily.
- 4:29
This kind of writing requires forming hypothesis. So you have to ask the model, "Okay, can the model do this?" And you want to test that. Can the model self-correct its responses?
- 4:42
Yes or no? Um, and so you start trying, like, forming the hypothesis. Next, you test certain assumptions that you make about the model. And if, uh, as you iterate more, you kind of, like, get new insights or like, "Oh yeah, the model's pretty good at this particular thing, but it's not super good at another thing to achieve
- 5:02
this task." So you gain more clarity about, like, what's, what's the strength and the weaknesses are.
- 5:10
Um, I wanted to start with, like, overall broad, like, writing principles, uh, because ultimately prompting is like writing, right? So the goal is to write prompts that clearly communicate the task objective while providing just enough constraints and guidance to steer the model towards producing high quality and relevant outputs.
- 5:34
And so there are, like, four to six kind of, like, writing guidelines that I think I found effective, um, especially working with, like, Claude. Um, so the re-
- 5:48
maybe to clarify how Claude is different from ChatGPT or GPT models, I think with Claude you have to almost treat it as, like, another human. So you have to, like, explain things to a [REDACTED:age] or, like, you have to be elaborate and, like, um- I will share like more examples on like how to do that.
- 6:12
Um, but I think that's like a distinguishing, um, feature from like GPT models from Claude. Um, [clears throat] so the first principle is clarity. Like, use simple, unambiguous language in your prompts, avoid confusing syntax or vague phrases that could confuse the model.
- 6:31
The second is conciseness. Keep prompts short and focused. Include only key information, uh, that the model needs.
- 6:41
Third is coherence. Logically structure the prompts with context at the beginning and clear task at the end.
- 6:49
Consistency, stick to similar formatting. If you use XML tags, use it consistently in the prompt. If you use certain like terminology, um, do not like kinda like put the model off distribution, basically.
- 7:06
So make it consistent. Uh, direction provide like genre, length, style, or any like guidelines, guidance to direct the model's response. Ground prompts with examples, sources. Make the model to code itself or like, uh, if you have like long document in the context, make it to quote from the document to support the argument
- 7:31
or may-- or help, help the models to form, support the arguments from like search results or like other supporting, um, contextual, uh, information. Uh, engaging, use diverse edge cases, examples, uh, very useful for few-shot prompting.
- 7:49
Um, now I'm gonna go through sort of like tasks that I thought would be interesting, um, and see how you can use Claude in those specific tasks. So the first case is obviously a recommendation system.
- 8:06
Um, as of last year, last year I made this project Interalia. Um, I used CLIP
- 8:16
to, um, kinda like, um... So I scraped a bunch of like
- 8:24
images and like, uh, clothing items from different brands. Um, and I used text as like an actual search kinda like, uh, engine. So you can say like, um, James Bond girl, and it can do the, the dress.
- 8:43
And in a way you get like results that are like dress in the style of James Bond or you can do like futuristic ethereal outfit, and it's like more like vibes-based search.
- 8:54
Uh, and you can like go and look at the shop itself.
- 8:59
There you go.
- 9:00
Yeah.
- 9:00
Can you explain CLIP?
- 9:01
Oh, yeah. Uh, CLIP is contrastive language to image model trained by OpenAI. It's open source, but it's basically, um, they provide embeddings for text and images. So you can like...
- 9:17
A- and the way it works here is that like you embed images and you embed, um, text result. And s- what you can do is you can like do cosine similarity to find the most similar, um, [clears throat] items based on your database for the user query.
- 9:34
I don't know if that's clear. Let me know if you have questions.
- 9:36
It's the same thing as multimodal embeddings?
- 9:40
Um-
- 9:40
It's a, it's one way to take image and text to become multimodal. There are different things for audio. CLIP is mostly for connecting text and images.
- 9:53
Yeah. Um, I think you can read the paper, CLIP, yeah, um, if you're interested.
- 10:01
So I was ke- I was thinking like, "Okay, how could I use Claude in this project to curate relevant recommendations based on the user's requests?" And that's the task.
- 10:14
So in a way, you have like user's input. Let's say dress in the style of Emma Chamberlain, blazer like in the Great Gatsby movie, an outfit with a futuristic vibe for the Met Gala.
- 10:26
And on the other side, you have like image to text database, um, with images and their labels, and the labels can be produced either by like the original source, uh, or you can use like multimodal model to like come up with labels based on the images.
- 10:41
And so the task with the Claude is to curate, like based on the labels from the images, decide whether this like item relevant. Should I recommend this to the user?
- 10:54
Is this accurate? Like, does it matches the, uh, user's pers-- like, can I personalize this? Um,
- 11:04
and so if you can look at the
- 11:08
very simple curation strategy, um, for the prompt. You can just like zero shot it. Like,
- 11:16
I need you to decide whether the item is relevant to the user query. Here's the user query. Um, here's the item description. Um, is that item relevant or should, should be recommended to the user based on the user's query?
- 11:31
Answer yes or no. Please write the answer in answer tags. Let's unpack this. First of all, Claude really likes XML tags. Really loves XML tags. I think like this is like number one mis-- not mistake, but like, um, one thing that people miss.
- 11:52
They don't like put anything in XML tags, and so they don't have like very high like good performance. So everything... Like yeah, if you love XML tags, then you should like put everything in XML tags. [laughs]
- 12:02
And, um, with XML tags, you should like be consistent. So what is user query item? And you can be very descriptive. I can share like more examples in a few, uh, later on.
- 12:14
And here, in a way you like... The way you interact with Claude is, like, you can see the language here. I need you to decide whether the item is relevant.
- 12:23
It almost feels like you talk to a human.
- 12:28
Um.
- 12:29
Question now.
- 12:29
Yeah.
- 12:30
So, uh, this XML thing recently only came out, like, maybe like a month or two ago from official Anthropic device.
- 12:38
Oh, really?
- 12:38
Is this something that was on, like, intentionally trained for or you discovered it after pre-training?
- 12:45
I think it was, uh... I mean, we tried to... XML, XML formatting was the first formatting that we, like, kinda like fine-tuned on. Like, um, later on, we discovered, you know, customers need, like, markdown or, like, nee- Claude needs to, like, use JSON, JSON formatting.
- 13:06
So like, we kinda, like, learned from customers, but originally it was, like, XML formatting. Yeah.
- 13:11
So is that mostly because of the fine-tuning or is that because you had a training set that had tons of XML stuff in it?
- 13:18
Uh, I think it's kinda both, yeah. Um, yeah.
- 13:22
Do you not need to close your XML tags?
- 13:24
Hmm?
- 13:25
You don't need to close your XML tags?
- 13:28
Close XML tags? [laughing] Yeah, I don't know. Um.
- 13:37
Does it, does it generate the HTML?
- 13:39
Oh, yeah, sorry. I had a mistake. Yes, I have. [laughing] Um, yeah. And so, like, one, one good thing about XML tags is, like, it's really, uh, easy to extract, right?
- 13:50
Like, the strings inside it. And Claude is pretty good at, like, um...
- 13:57
I can, I can, like, say, like, "Don't put anything in XML tags." So sometimes, like, Claude will, like, say, "Here's information," blah, blah, blah, blah. But then if you ask, "Just, like, write the answer in, like, this tags," Claude will not put any additional information, which is one of the most annoying thing with, like, language models.
- 14:14
So here is the results, uh, that I put. Um, this is through Claude.ai interface. Um, and so yeah, you can see, like, is this item relevant? Uh, says no.
- 14:27
Is this item relevant? Yes. But I don't think that's, like, a hundred percent, like,
- 14:34
um, you know, perfect system. So it's very, like, zero-shot. So basically, you can, like, iterate, and we will try to iterate more on this.
- 14:45
Um, strategy number two is that, like, when, uh, you put-- you ask the model to take some time to think whether the item is relevant or not, uh, in thoughts tags based on the criteria above.
- 14:58
Um, and you kinda let, let the model think, um, a little bit more, um, with its reasoning. And then, uh, this is, like, basically chain of thoughts.
- 15:11
You can also add, like, criteria. Uh, so as a part of your critique, consider the following criteria. And so if you wanna, like, steer the model on, like, does the item match the specific attributes requested by user?
- 15:24
Like, help the model to, like, think kinda, kinda like think through, like, what means... What does it mean to, like, recommend an item to the user? Does the item match the season or were the conditions matching in the user's query?
- 15:37
For example, you should not recommend winter coats during summer seasons. So, like, in the criteria, you can, like, give more examples, more elaborate examples.
- 15:48
Um, another thing that you iterate on is, like, not just, like, give answer yes or no, but you can, like, based on your critique, score whether the item should be recommended or not, where one is least to be recommended and ten is highly recommended, and put the final score in score tags.
- 16:08
And so how does it work? Uh, so, um, here, like, user query James Bond blazer, um, item which I took from, um,
- 16:19
I think it was some brand. Um, and Claude would, like, start, like, thoughts tags. Um, overall it seems, like, very relevant, and the final score is nine.
- 16:33
And here's another example. I want a dress in the style of The Great Gatsby movie. Um, here's the item, braided cord cropped waist coast. And the critique is basically, uh, the item is not appropriate for the user's needs based on the context clues in the query.
- 16:49
Um, it doesn't, like, you know, match the attributes of The Great Gatsby movie. It, like, tries to, like, have, like, some reasoning. And so the score is two.
- 16:58
And you can be a little bit more elaborate. This is, like, very simple, like, iteration, um, on, on that. Do you guys have any questions? Yeah.
- 17:07
So one interesting thing that I saw, I mean, there was the XML tags-
- 17:10
Yeah
- 17:10
... now here that weren't closed. My friend, he is not a native English speaker. His prompts are always in, in very kind of funny English, but he structures them really well and they work really well despite the English being-
- 17:24
Mm-hmm
- 17:25
... very incorrect, right?
- 17:26
Right.
- 17:26
Why? Why does that work?
- 17:28
I think it's just the models are, like, pretty good at, like, knowledge transfer between, like, languages or, like, can infer very well on, like, the user's intent. Um, yeah.
- 17:42
I don't have, like, pretty clear answer.
- 17:43
It's all probabilistic, so that text with one syntax mistake looks close enough to the text with the right syntax. So the probability of the real answer is close in both cases.
- 17:54
Right.
- 17:54
Like, yes, I don't know.
- 17:56
Yeah. Yeah.
- 17:57
In this particular example-
- 17:58
Mm-hmm
- 17:58
... I'm curious on whether you see any bias with the score.
- 18:02
Mm-hmm.
- 18:03
In other words, if you were to look at the distribution of scores-
- 18:05
Right
- 18:05
... would it be a normal distribution?
- 18:07
Yeah. This is an interesting question. Like, this is one question that we ask in our research settings. Like, one thing that we're trying to understand ... Like, we have a research group, uh, called societal impacts, and one thing, uh, that we, we are trying to understand now is, like, when you summarize, like, news articles and you try
- 18:27
to evaluate, like, the bias with kind of the distribution. And I feel like this is, like, research active. Like, yeah, I think it depends on the task, um, really.
- 18:37
I did not test on this [laughs] literally versus yesterday, uh, prompting. [laughs]
- 18:45
Cool. Um, the second task, um ... So Claude is known for 100K co- context, uh, size, which is the entire book of The Great Gatsby can, like, put into the context and you can, like, ask the model, uh, summarize the book or, like, uh, ask some tasks based on the huge context.
- 19:07
And this is, like, m- basically time test compute, um, thing. And so with long context, um,
- 19:19
let me see. The way you can use long context can be in different ways. Like, one way is, like, you put multiple documents and try to summarize or, like, uh, retrieve information based on the documents.
- 19:33
Another way to use long context is to have a bunch ... a huge few shot prompt.
- 19:40
And so as you know, like, chain of thought, um, technique relies on the stated reasoning faithfully reflecting the model's actual reasoning. And in one of the recent papers, we found that it's not super ...
- 19:54
or, like, it's not always the case. So
- 20:01
doesn't ... So basically what it means is that, like, if you ask the model do a chain of thought, it might not necessarily attempt to, you know, chain of thought, uh, to produce the final answer.
- 20:14
It might just, like, ignore it or, like, uh, not take any account. So it's not ... We, we call it, like, unfaithful basically. It's not super faithful.
- 20:25
And so we propose in this paper, um, like, decomposition-based methods can actually achieve, like, strong performance on specifically question answering tasks, sometimes approaching that of chain of thought performance while improving the faithfulness.
- 20:45
Do you guys have any questions?
- 20:47
I didn't understand that. Sorry.
- 20:48
Okay. Yeah.
- 20:49
What is faithfulness here? Is the faithfulness to your prompt or to your-
- 20:54
Um, faithfulness is, um ... Yes, to your prompt. Yeah.
- 21:01
What's decomposition? What's decomposition again?
- 21:03
Yeah. Uh, let me explain what decomposition is. So here's the graph from the paper. Uh, we have, like, three methods. First is a chain of thought method, which is like, uh, here's the question: Could Scooby-Doo fit in a kangaroo pouch?
- 21:20
Uh, there are two choices: A, yes; B, no.
- 21:24
Chain of thought prompt saying, like, [clears throat] "Let's think step by step," gives the reasoning. Um, the human ask the follow-up questions based on the above, what is the single most likely answer choice?
- 21:39
And, and the model says the correct answer choice is B,
- 21:43
right? The chain of thought decomposition is when you decompose a question ... when you can ask the model to decompose a question into, like, multiple sub-questions so that each sub-question are kind of independent from each other.
- 21:58
Because in chain of thought, like, you have one, two, three, you know, like, they kind of, like, can influence each other, right? Like, in decomposition you kind of like, um, y- y- you decompose and you, like, put each s- sub-question in the independent context.
- 22:14
So in a way, it kind of, like, reduces the bias.
- 22:19
Um, and so in this, um ... Uh, let's, let's see here is, like, sub-question one: Uh, what type of animal was Scooby-Doo? The answer from the model, Scooby-Doo is a fictional character.
- 22:32
Another sub-question f- for, uh, for the assistant for Claude, how big is an average kangaroo pouch? And you ... And what you can see is that, like, each sub-question is kind of, like, self-contained.
- 22:43
It's very atomic self-contained question. Um, and so you have, like, multiple sub-questions like this, and then what you do is you recompose. So, like, you, like, put sub-question, answer sub-question, answer sub-question, answer into, like, one context and ask the model, based on the above what is the single most likely answer choice?
- 23:05
The correct answer choice is B. Yeah.
- 23:08
Um, i- in chain of thought, like, in the system prompt or whatever the user's prompt is, like, we mention let's think step by step.
- 23:15
Mm-hmm.
- 23:16
Uh, w- what do you do for the decomposition? Is there, like, a similar,
- 23:22
you know, input to the model-
- 23:24
Mm-hmm
- 23:24
... to make it decompose into multiple questions?
- 23:26
Yeah. Uh, I can share the prompt, um, in a few slides, uh, on this. Um, but, uh ... Yeah. Any other questions?
- 23:35
Can you show the graph again? Now I know it is-
- 23:37
The, this graph?
- 23:38
Yeah.
- 23:49
Um, yeah. Let's look at the prompt. Um, very hard to see, but I'll share the slides. Um, let's ... Um, I, I'm, I'm gonna give you, like, legal context, like, legal question.
- 24:06
Let's say you have a question on, like-[clears throat]
- 24:09
A, a legal question and you ask us like, whi- which of the following is the most persuasive argument that a person is liable to the creditor under the terms of the agreement and here's the context?
- 24:20
So that's the question, basically. And so you have like choices for the model. So this is like multiple choice question and before that you have like a huge few shot prompt.
- 24:33
Um, and basically here, uh, to answer your question, like it says, "I'm going to give you a question. I want you to compose into a series of sub-questions. Each sub-question should be self-contained with all the information necessary.
- 24:48
Um, this is really important, blah, blah, blah. Uh, make sure not to decompose more than necessary. Um,
- 24:59
be concise, blah, blah. Please put each sub-question in like these tags but include the numbers corresponding to each tag."
- 25:08
So, um, and the model says, "Yes, I understand. Uh, you have a question,
- 25:16
um, multiple choice answers," and the model provides sub-questions for you and then what you do is that you try to answer the first sub-question and you give it to the model.
- 25:29
You try to answer the second sub-question, you give to the model. Third sub-question, you give to the model and then later you say like based on everything above like you give all the context, um, answer me the question.
- 25:44
Um, the correct answer C. Yeah. And so this is like very similar in the legal context. You have sub-questions like what is consideration in contract law?
- 25:59
Blah, blah, blah. And you have-- you can have like another model to like sample here. You can have like another model to answer this. It doesn't necessarily should be like one model.
- 26:08
Um, and then there's like another sub-question and here's the answer.
- 26:17
Yeah. Do you guys have any questions? Um,
- 26:26
the second thing that I wanna talk about is how to use Claude to do evaluations. Like evaluating like Claude on like long context ability. Let's say you have a lot of like documents and you want to understand how good Claude is answering questions based on the document
- 26:51
or is it, uh, is it able to answer like the questions not just like from its pre-trained knowledge but like based on the document itself.
- 27:02
And so, um, I'm gonna give you example that we did at Anthropic. Um, multiple choice QA, um, evaluation design. So our goal was to-- with this exercise to evaluate techniques to maximize Claude's chance to correctly recalling a specific piece of information from a long document and so the document that we chose was a government document
- 27:28
that contains like a bunch of like meeting transcripts, different departments and we also chose the one that was like, uh, from this year July thirteens, uh, which is like way after Claude's, um, training data cut off so that you don't like, um, you have the document that does not have in the pre-trained knowledge or something.
- 27:53
And so what you're trying to do is like now you want to use Claude to generate question-answer spares. Um,
- 28:05
you, you in a way like you create like data, data set based. You use language models to create like data sets and so the way you do that is that you split the document into sections and use Claude to generate like five multiple choice questions for each section.
- 28:21
Each with three wrong answers and one right answer and if you do that you then reassemble like randomized sets of those sections into like long documents that you could pass them to Claude and test its recall, uh, of their contents.
- 28:38
This is very meta. Let me know if you have questions. Yeah.
- 28:42
Makes sense.
- 28:44
Um, so here's a prompt to generate multiple choice questions. Um, I a- we ask like please write five factual questions for this. Um, [clears throat] some guidelines at the end.
- 29:01
Um, and basically we test different strategies, prompting strategies just asking Claude, give Claude two fixed examples of correctly answered general knowledge and, um, that are unrelated to the government document.
- 29:18
Um, providing two examples and providing five examples of correctly answered questions and we tested the strategies, uh, on different settings like one is containing the answer positioned at the beginning, the end or the middle in the input and we tested with like seventy k and ninety-five k token documents.
- 29:43
You can look at the prompt and more specific how we did this in our blog post
- 29:51
but basically the result is this. Uh, here we see that um-
- 30:02
What is the metric?
- 30:05
Yeah. The metric was um, to... Let's see
- 30:15
Like, basically, how many, how many times, like, Claude has correctly answered the question. Um, right. And so...
- 30:31
Yeah, sorry. Uh, basically what we find is that, like, for document Q&A, asking the question at the end of the prompt performs a lot better than asking at the beginning.
- 30:42
You can see it here. Uh, pulling relevant quotes into, like, critique or, like, thoughts tags is helpful. Um, it's like the small cost to latency but improves accuracy.
- 30:58
Uh, and we tested on, like, both Claude and Claude Instant.
- 31:03
Um, and it seems like you can boost way better performance from, like, Claude Instant, um, than Claude 2.
- 31:15
Basically, the idea is that, like, if you wanna use long doc Q&A, put the instructions at the end of your prompt. Yeah. That's, like, the result of this. [laughs]
- 31:27
Wait, what was the-- I didn't catch, what was the scratchpad in the-
- 31:32
Uh, oh, yeah. Like, you just ask the model to, like, put thoughts in, like, thoughts tags before answering the question, so it has, like, more reasoning-based approach. Yeah.
- 31:42
Can you go to the table again? Sorry, I just wanna-
- 31:44
Yeah. Sorry. I'm, uh... [laughs]
- 31:47
So just for the averages for me. But, like, the outcome was basically putting it at the end matters more than all the other-
- 32:02
Yeah
- 32:03
... optimizing strategies.
- 32:04
Right. Yeah. Um, yeah. In a way, this is, like, an example to show, like, how to use Claude to generate a dataset that you can, like, evaluate and, like, you can use it for, like, evaluation, basically.
- 32:27
Yeah.
- 32:28
So this has to do with you putting your instruction at the end of the prompt.
- 32:32
Mm-hmm.
- 32:32
Uh, first are there theories on, like, why, why specifically instruction should be at the end?
- 32:37
Mm-hmm.
- 32:37
And are there any things like... Do we have any understanding of,
- 32:43
like, are there certain things at the beginning of the prompt that still might be weighted or is it, like, this sliding scale that, like, the further in the beginning of the prompt, like, the less attention it gets or influence it has?
- 32:53
Yeah. I think that's basically the hypothesis.
- 32:55
Okay.
- 32:55
It's like the, you know, it, it's like the distance. It's like the model attends more to the end of the prompt-
- 33:03
Okay
- 33:03
... uh, than the beginning.
- 33:04
It's not like people were... There's exact [laughs]
- 33:06
I think there was a paper saying, like, it just forgets in the middle or something.
- 33:09
Oh, okay. Yeah.
- 33:10
Um, yeah, I think this is the problem with, like, long context that we're trying to fix or something. Yeah.
- 33:16
So to, to follow on that question, you are saying that there was a paper that said that it remembers the beginning and end and kind of forgets in the middle.
- 33:24
Yeah.
- 33:24
But what you're saying is for Claude 2 it seems to do best if you give it at the end.
- 33:29
Yeah.
- 33:30
So that paper doesn't apply to Claude?
- 33:32
Um, I did not read that paper like-
- 33:37
No, I'm just curious. Like, what you're saying is you're finding it, at least for Claude-
- 33:42
Right
- 33:42
... the end part gets more attention.
- 33:45
Yeah. For, like, a specific task is, like, a long context, uh, Q&A for, like, long documents. Yeah. Um, yeah, we have not tested on other tasks to my knowledge.
- 33:59
Um, so the prompts you showed were using regular prose and then the XML tags.
- 34:05
Mm-hmm.
- 34:06
Um, I think that's also what's in the Anthropic docs. Have you guys ever done experiments on, like, that kind of format versus markdown-
- 34:16
Mm
- 34:16
... versus everything is in XML? Do you have any thoughts on that?
- 34:20
Yeah. So, um, in general I think, I think it's because
- 34:30
markdown was kinda like... There's not that much of like...
- 34:37
I don't know. It's, like, best in XML tags. Like, I'm, I'm, like, thinking the-- I've, like, tried Claude to like, you know,
- 34:45
use, like, JSONL or, like, uh, use markdown, but sometimes it's like, you know, it's not as good as, like, XML. With XML it's almost a hundred percent accuracy. Yeah.
- 35:03
Let's see. Yeah. Um, let's go to another task, um, which is, like, you can use language models to, like, auto-label basically anything. Um, so one of the examples that we did last year, um, we asked Claude to categorize the labels for the clusters.
- 35:28
And so, um, this was for the paper but the approach was very simple. We have a bunch of like, you know, texts and we embed them in UMAP. Um, and we do, like, KNN clustering
- 35:46
and for each cluster... Sorry, K-means clustering. And for each cluster select, like,
- 35:56
for each cluster aggregate all the, you know, little, like, statements, uh, claims and we ask the model to come up with a category for this cluster.
- 36:08
So that's the approach and you can look at the other
- 36:15
labels Uh, labels here are not super good because we used Claude 1.3 at that time. Claude 2's supposed to be, like, way better at this. Um, but this is, like, you know, cached.
- 36:27
It was, like, last year. Um, where's my slides?
- 36:35
And so one thing that you can do with this kind of task,
- 36:45
we call it self-consistency. You can generate N samples for the question. Um, so let's say you have a question like, "How do you label this cluster?" And you generate independently N times, and you can ask just, like, come up with, like, one category.
- 37:07
Um, well, this method is mostly useful for, like, quantitative, like, if you have, like, a math question and you sample, like, different... like, sample multiple times and come up with the answer.
- 37:19
Um, like, the most common answer is, uh, the one that you select for the final answer, and this is called the majority vote.
- 37:29
Another technique that you can use is, like, um, have, like, two generated samples and ask another model to evaluate whether those samples are consistent or not. And if the samples are consistent, the-- well, you gain more confidence that this is correct, right?
- 37:44
And if it's not consistent, you just, like, deselect.
- 37:50
Um, another thing that you wanna do with Claude is, uh, if you--
- 37:56
If Claude is kinda, like, misses the nuance, especially for, like, categorizing a lot of labels and you have, like, a, a lot of, like, categorizations, um, [clears throat]
- 38:08
you can add contrasting conceptual distinctions in your instruction, and you can do it in multiple ways. One way to do it is, like, you provide bad example. Let's say, like, "Here is a very bad category, and you should never come up with it because this is, like, too narrow or, like, too general, and this is not what
- 38:27
I want." Uh, like, give, like, contrasting, like, examples. Uh, vary the context. Use examples in different contexts and settings. Um, not just, like... Just, like, have, like, more diversity.
- 38:40
Like, diversity is, like, um, an... Uh, uh, the more diverse, like, few-shot prompt examples, the, the better.
- 38:47
Use analogies and metaphors. Um, if the concept is, like, too hard to understand for the model, try to, like, decompose and, like, bring analogy. Um, point out, like, common misconceptions, um, especially for, like, categorizing, like, [clears throat]
- 39:05
let's say, what is false presupposition, right? Like, uh, point out the common misco-misconception and, like, clarify, like, why this is, like, incorrect. Like, provide examples, like, specifically show why common mis-misconception is wrong.
- 39:26
Uh, yes. Do you guys have any questions? Yep.
- 39:31
So for the, the label, the goal is you, you run K-means to get groups and then you wanna give a name to the group. Is that the label matching?
- 39:41
Just to clarify.
- 39:43
Um, I cannot super hear, but, um-
- 39:45
Yeah. It... So the goal here is you're, you're running, uh, a clustering algorithm to get groups.
- 39:50
Mm-hmm.
- 39:51
And then the, the task of the LLM is to put a name on each group?
- 39:55
Yeah.
- 39:57
Okay. Okay.
- 39:57
Like, come up with a category. Like, um, yeah, come up with a category or, like, classify, uh, [clears throat] like, label that cluster basically.
- 40:09
Um, so here is, like, ba-very basic, like, tips and strategies with Claude API. Um, number one is formatting. Um,
- 40:18
like, Human: Assistant is, like, what Claude loves, and if you misses, you miss it, like, you'll get, like, very, very terrible results. Uh, new line, new line, Human, new line, new line, Assistant.
- 40:31
Um, yeah. Uh, you can also put words in Claude's mouth to, like,
- 40:39
kinda, like, say, like, "Do you understand it?" And you can, like, put in the, um, Claude's mouth, "Yes, I understand it," in a way to, like, you know, put, put the model i-into these modes.
- 40:54
Have Claude repeat instructions back. Um, you can say, like, "Do you understand the instructions?" Um, and you can put, like, Assistant, "Yes, I understand instructions,"
- 41:09
blah, blah, blah. Uh, to reduce hallucinations, like, let Claude
- 41:17
hedge and, like, say, like, "I don't know," or, like, uh, [laughing] "I don't have enough information or, like, context to answer the question."
- 41:29
Um, here is another thing. Um, if you have, like, generate direct quotes, if you have, like, a document or, like, um,
- 41:40
a long document in the context, um, make Claude to say, "Find appropriate quotes," but also say, like, um, "If there are no quotes in this document that seems relevant to this question, please just say, 'I don't find any relevant quotes,'" so that it doesn't make up
- 42:00
or fabricate new quotes. Uh, how to give good examples. Um, are the examples similar to the ones you need to classify? Are the examples diverse enough for Claude not to overfit to, to the specifics?
- 42:16
Equally distributed among answer types. Don't always choose option A, but, like, y- kinda, like, have the diversity.
- 42:25
Um, yeah. I get a lot of quest-
- 42:30
Oh.
- 42:30
Yeah.
- 42:31
In the first example you said the formatting was the, the bad part. I can't see exactly what's wrong with the formatting in that.
- 42:39
Oh, formatting in a way... Oh, here?
- 42:42
Yeah.
- 42:43
Um, I think they didn't put like new line, new line. Pretty sure.
- 42:54
Oh, the... Here or here. Sorry. Here, you put like human assistant inside the XML tags. You only need-- You only have to use human assistant
- 43:05
as like tokens to like sample, but you should never put it in like, like inside the context itself. Either use like user and like other like, um, you know, um,
- 43:18
words like user AI or something, or like H or A, but you should never use human assistant. Human assistant is like very special, special words.
- 43:29
If, if that didn't have the XML tags, would it, would it be okay?
- 43:33
Um, it would be okay, but you would like make... You should like have human and then assistant in between, and then human assistant and then another human and assistant, basically.
- 43:48
Uh, yeah, formatting is like human assistant, human assistant. You should never have like human, human, assistant, assistant or something. Uh, that's bad. Um, yeah.
- 44:01
I think we have like more extensive, uh, explanations in the API docs, if you can look at it.
- 44:10
Yeah, I get a lot of questions like what the future of prompt engineering is. Um, and I think the answers are pretty clear. Like prompting will stay. We'll just ask like more complicated, nuanced questions or like tasks for the model.
- 44:25
Um, prompt engineering is a, we will like-- We're moving towards the world where we'll have like more and more synthetic data generation. And so I'm pretty optimistic about like mo- using models to like generate like diverse sets of like datasets.
- 44:40
Um, you can also use language models to write like evaluations. Um, so you use prompting to do that. Um, reinforcement learning from AI feedback, um, is an alternative to like reinforcement human-- from human feedback, which is like a little more scalable.
- 45:01
But basically, you ask the model to revise its own, uh, responses in the process. So you give the model-- Y-you like ask the model to like self-reflect or like self-revise.
- 45:13
Um, and so you use prompting in that process to do this.
- 45:18
And especially like prompt engineering will become like a, a standard part of like product development. I feel like, um, things that we did in Claude products such as like auto-generating titles.
- 45:33
Like these things was never like done before, like cl- cl- uh, before like large language models. And so you can like create delightful mini UX experiences, uh, such as like that using just prompting or something.
- 45:48
And you can like have personalization. Uh, maybe you can embed all the users conversations and like suggest like new topics for the conversation. Um, and you can use models to do that.
- 46:03
Um, and the most like interesting thing is like, uh, finding most optimal prompts for specific tasks. Maybe you want to like minimize the number of tokens to get the highest accuracy for the task.
- 46:13
Um, yeah. Uh, here are some resources. Uh, we just, uh, launched Anthropic Cookbook with like certain like demos on research, uh, on retrieval and search. Um, we have prompt design guide w-- in API, uh, book.
- 46:30
Um, you can also read our, the papers that we publish. Uh, oftentimes we have like appendix with like all the prompting that we do.
- 46:39
Yeah. Thank you so much. And, uh, if you have any questions, let me know. [audience clapping]
- 46:50
Uh, yeah, we have five minutes for questions. Charles is coming up. We also have water. Thanks to Sean, um, for bringing in some water so, uh, get hydrated people.
- 46:59
But five minutes for questions for anything about prompt, large prompt engineering.
- 47:08
Yep.
- 47:09
So have you tried these with different things besides Claude, like other...
- 47:14
Have you had similar kind of results? 'Cause you're talking about Claude in this one particularly. Like maybe like with OpenAI or other-
- 47:24
Yeah. I think I'm most, um, experienced with Claude because I use it like every day. Um,
- 47:32
less experience with GPT. Uh, I did not look carefully, to be honest, at their like API docs. Um, but it seems like the strategy is a little bit different.
- 47:44
Yeah.
- 47:45
Okay.
- 47:45
They don't have like formatting as VR, let's say. Yeah.
- 47:49
Uh, great, great presentation and, and that work was very impressive.
- 47:54
Thanks.
- 47:54
Uh, have you seen this prompt reader that came out recently? Is that, uh, where you basically using LLM to improve the prompt, uh, considering the outcome? Uh, do you see that in the-- inside the future, say, optimal tasks?
- 48:06
Is that what you mean or do you see other directions for, uh, optimizing prompts?
- 48:12
Yeah. I think, uh, that's, that's actually one of the directions too. Like, um, rem-- Don't remember
- 48:20
how, what was that paper called? Like LLMs as like optimizers, I think, right? Um,
- 48:26
but yeah, I guess like, um- In a way, there are, like, certain tasks that the models are, like, not good at currently. Like, for example, like, self-correction. Like, the models are not really good at, like, self-correcting their, like, answers.
- 48:40
And, like, can you find, like, a prompt that is, like, pretty good at it or, like, um, other tasks that you want? Yeah.
- 48:51
So I'm curious-
- 48:52
Yeah
- 48:52
... about what techniques your team is using for actually, like, evaluating the quality of the responses. That's something that a lot of teams kind of struggle. Like, how do we actually automate this?
- 49:01
Yeah. I think, um, depends on the task. Sometimes we just, like, have to look manually, qualitatively, uh, at outputs. Um, sometimes you, let's say, um,
- 49:14
you wanna evaluate, you know, how much does the model refuses and if it refuses in a relevant context or not. And so, uh, you use, you know, generated answers, and you categorize refusals [laughs] in different categories and use the model to categorize that.
- 49:35
And so you just, like, see the rate, um, yeah.
- 49:40
I can think of that example. Yeah. It depends on the task. Some tasks are, like, you know, um,
- 49:48
for, like, hallucinations, you ha- actually have to, like, look yourself or something. Yeah.
- 49:53
Yeah.
- 49:54
So you mentioned, uh, something about clustering, uh, different kinds of questions that we need to ask different categories and, uh, giving it a label. Uh, so have you tried using OpenAI functions which has, uh, some enumeration limited times with the predefined classes or intents, user intents and have you seen if that improves the accuracy of the classification
- 50:18
when you try to-
- 50:20
Of with OpenAI?
- 50:22
Of, of any kind of function calls. Uh, the OpenAI function call is a model.
- 50:26
Uh, if it's any kind of, uh, model that takes structured data that can, uh, run further model functions.
- 50:34
Yeah. Yeah. I think, uh, um, I wouldn't say m- too much about this, but I actually didn't-- have not, like, extensively used function calling from OpenAI or, like, other models.
- 50:45
It limits the, the classification if you don't-
- 50:48
Right
- 50:49
... use it.
- 50:51
Yeah. Cool. Yeah. Yes.
- 50:54
So you mentioned something about, you know, using an LLM to generate titles for customers. How did you actually try to evaluate if the titles are relevant-
- 51:04
Mm-hmm
- 51:04
... and actually consistent, like-
- 51:06
Mm-hmm
- 51:06
... beyond human feedback?
- 51:09
Yeah. I think, uh, one int- Uh, actually this is an interesting question. Like, I worked on the auto-generating titles for Claude.ai, and, um, one thing that I asked Claude is to be like an editor, like, have an editorial taste.
- 51:25
And what we did is actually we took previous titles, and we put in the context to generate a new title. And so in a way, it's, like, a little more consistent to, um, what the style of the user is.
- 51:41
Yeah.
- 51:41
Would you be able to share the recall precision in terms of catching defects if they do happen or-
- 51:47
Uh, yeah. I'm not sure if I can share that. Uh, yeah.
- 51:51
Okay. Was it reliable enough to use in production?
- 51:54
Uh, yeah. We, we use this in production. I can, like, show you, like, Claude.ai interface.
- 52:00
And, uh, one thing that we changed recently is that, like, if you have, like, pretty, like, short, like, um,
- 52:08
you know ... Sometimes you don't have-- you don't need, like, LLM to come up with a title. You just take... If, if the prompt is, like, very short, you just, like, um, use the, like, the first, like, words.
- 52:19
Um, but here, yeah, like, I don't know. Um,
- 52:23
let's see. Here is introduction, but then, like,
- 52:29
recommend some books. Um, yeah, I don't know. Uh, [laughs] yeah.
- 52:40
Uh, I have a quick question.
- 52:41
Mm-hmm.
- 52:41
Did you cover the difference between Claude and Claude Instant?
- 52:45
Um, no, but I can tell. Uh, so Claude-
- 52:51
For those who don't know, she trained Claude Instant. And I'm like, "You might as well cover the thing that you did." [laughs]
- 52:56
Yeah. [laughs] [laughs] So let's look at the ... Is there some docs on this? [laughs]
- 53:02
When did you guys announce the 1.2 thing?
- 53:05
When did we announce? Uh-
- 53:10
August 9th
- 53:11
... August 9th? August 9th. Um, basically Claude 2 is a larger model. Uh, it's a little bit smarter. It's, like, smarter than Claude Instant. Claude Instant is way cheaper and way faster.
- 53:25
But Claude Instant is better than Claude Instant one, um, in the, like, more, like, reasoning-based tasks. So it's b- way better at math. Uh, it's way better at code.
- 53:36
Um, other benchmarks are, like, pretty similar, but I think we specifically trained Claude Instant to be good at, like, math and code.
- 53:47
Um, and it's, uh, way better at, like, red teaming. Um, like, automated red teaming evaluations, so it's more robust to, like, jailbreaks.
- 53:57
Um, yeah. I really like this model. You guys should use it.
- 54:03
Good. [laughs]
- 54:03
Yeah.
- 54:05
Thank you.
- 54:05
Yes.
- 54:05
We exposed, exposed my ignorance. So when you talk about training, like, when you train this Claude Instant, was that ta- fine-tuning or was that something different?
- 54:13
Uh, yeah, fine-tuning.
- 54:15
Okay.
- 54:15
Yes.
- 54:15
Last question.
- 54:19
Can, can you say more, I guess, red ... Like, what is red teaming and how does that work?
- 54:23
Yeah. Red teaming is interesting concept. It's, um, basically you ... Like, the models are pretty, like,
- 54:32
uh, vulnerable to, like, certain, like, jailbreaks. Um, so sometimes, let's say, like, a very simple example. Like, can the model give you instructions how to build a bomb? And so we, we consider it as a jailbreak.
- 54:46
And so the goal is to, like, uh, in that cases, like, the model should, like, refuse or do not, like, provide any additional information in case of, like, um, unsafe, like, prompts or something like this.
- 55:00
And so this is, like, the internal evaluations that we have. Um, you can read in the model card that we ha- we launched in Claude 2 how we specifically do that.
- 55:11
Uh, but it's basically the amount of, like, um, how robust the model is to, like, those jailbreaks. Yeah.
- 55:20
Cool.
- 55:21
Thank you. Thank you very much.
- 55:22
Yeah. Thank you. [audience applauding]
- 55:28
Yeah. Uh, so-