Writing Principles for Task-Tuned Prompt Engineering

Talk transcript

1. 0:01

   My name is Karina. Um, recently I've been working on Claude, which is, like, a large language model trained by Anthropic. And, um, most recently I was working on, like, reducing hallucinations, how to make Claude self-correct its answers, and many other features that went into a Claude 2 launch.

2. 0:26

   And so I'm gonna talk today about the writing principles for task-tuned prompt engineering and kind of sort of, like, help you if you want to use Claude API, help you to guide what's the best practices and tips that I found and we found most effective.

3. 0:45

   So first of all, um, I would like to talk about why prompting is hard. And to understand why prompting is hard, we should understand what prompting is in the first place.

4. 0:56

   So these models estimate the probability of each subsequent word given the preceding words. So in a way, a well-crafted prompt can increase the probability of generated desired and accurate, um, phrases.

5. 1:12

   Due to attention mechanisms in large language models, the models can focus on specific parts of the input text. And so effective prompts ensure that the attention is directed, uh, appropriately for desired outputs.

6. 1:27

   And so it's important to incorporate, like, task-specific keywords and context and examples within the prompt to activate the relevant portions of the model's internal knowledge. And lastly, like, you know, prompting leads to better results because without the need of, like, computational, like, other compute, you just, like...

7. 1:49

   W- without, like, model retraining. So, uh, you can, like, leverage inference time to test compute for this. And so, like, why prompting is hard. And I think I found based on, like, my conversations with customers and, like, developers, I think prompting is hard because of, like, three different reasons.

8. 2:06

   First, people know, humans know what they want, but they don't know how to get the best performance from the model, and I think that's what we're gonna focus on today.

9. 2:17

   The second reason is that they vaguely know what they want, but they don't know how to explain the best to the model, and so the model gets confused what the human wants from the task.

10. 2:31

    And the third reason is, like, they don't know what they want. So the humans don't know what they want, so it's pretty bad. [laughs] Um, and it's hard for the model to understand.

11. 2:40

    So basic strategies that you can, like, if you find yourself, like, you kind of vaguely know what the task is, just provide a bunch of examples, and the model will be good at inferring what you're trying to do just based on, like, examples.

12. 2:57

    And the examples may be, um, should be diverse and should en- encapsulate a bunch of, like, edge cases.

13. 3:06

    Try to explain as you would have to explain to a [REDACTED:age] or, like, very, you know, in a very simple terms. And I think what I found is that, like, as you have to, like, be able to, like, iterate a lot and, like, spend a lot of time just prompting.

14. 3:24

    And in a way, based on my experience, for example, as, like, research engineer, I spend the majority of pairing with people just collaborating on the prompts.

15. 3:35

    Um, in the past I was thinking, like, how my main experience with the word prompt was only in the creative writing classes. Uh, I graduated from Berkeley a- and I took some, like, creative writing classes, and we usually have, like, exercises, like prompting exercises, right?

16. 3:52

    And so we often forget that prompting language models is actually an act of creative writing. And I see people get annoyed, like, why their prompts just don't work. But in most cases, I think it just means that people lack some kind of, like, originality or creativity to think, uh, of, like, new novel ways how to make it

17. 4:12

    work. Um, I recently wrote a blog post about the cultures of writing, and one of the points that I'm making in the blog post is that prompting becomes like a form, a new form of writing for any research engineer and, like, scientist who engages in daily.

18. 4:29

    This kind of writing requires forming hypothesis. So you have to ask the model, "Okay, can the model do this?" And you want to test that. Can the model self-correct its responses?

19. 4:42

    Yes or no? Um, and so you start trying, like, forming the hypothesis. Next, you test certain assumptions that you make about the model. And if, uh, as you iterate more, you kind of, like, get new insights or like, "Oh yeah, the model's pretty good at this particular thing, but it's not super good at another thing to achieve

20. 5:02

    this task." So you gain more clarity about, like, what's, what's the strength and the weaknesses are.

21. 5:10

    Um, I wanted to start with, like, overall broad, like, writing principles, uh, because ultimately prompting is like writing, right? So the goal is to write prompts that clearly communicate the task objective while providing just enough constraints and guidance to steer the model towards producing high quality and relevant outputs.

22. 5:34

    And so there are, like, four to six kind of, like, writing guidelines that I think I found effective, um, especially working with, like, Claude. Um, so the re-

23. 5:48

    maybe to clarify how Claude is different from ChatGPT or GPT models, I think with Claude you have to almost treat it as, like, another human. So you have to, like, explain things to a [REDACTED:age] or, like, you have to be elaborate and, like, um- I will share like more examples on like how to do that.

24. 6:12

    Um, but I think that's like a distinguishing, um, feature from like GPT models from Claude. Um, [clears throat] so the first principle is clarity. Like, use simple, unambiguous language in your prompts, avoid confusing syntax or vague phrases that could confuse the model.

25. 6:31

    The second is conciseness. Keep prompts short and focused. Include only key information, uh, that the model needs.

26. 6:41

    Third is coherence. Logically structure the prompts with context at the beginning and clear task at the end.

27. 6:49

    Consistency, stick to similar formatting. If you use XML tags, use it consistently in the prompt. If you use certain like terminology, um, do not like kinda like put the model off distribution, basically.

28. 7:06

    So make it consistent. Uh, direction provide like genre, length, style, or any like guidelines, guidance to direct the model's response. Ground prompts with examples, sources. Make the model to code itself or like, uh, if you have like long document in the context, make it to quote from the document to support the argument

29. 7:31

    or may-- or help, help the models to form, support the arguments from like search results or like other supporting, um, contextual, uh, information. Uh, engaging, use diverse edge cases, examples, uh, very useful for few-shot prompting.

30. 7:49

    Um, now I'm gonna go through sort of like tasks that I thought would be interesting, um, and see how you can use Claude in those specific tasks. So the first case is obviously a recommendation system.

31. 8:06

    Um, as of last year, last year I made this project Interalia. Um, I used CLIP

32. 8:16

    to, um, kinda like, um... So I scraped a bunch of like

33. 8:24

    images and like, uh, clothing items from different brands. Um, and I used text as like an actual search kinda like, uh, engine. So you can say like, um, James Bond girl, and it can do the, the dress.

34. 8:43

    And in a way you get like results that are like dress in the style of James Bond or you can do like futuristic ethereal outfit, and it's like more like vibes-based search.

35. 8:54

    Uh, and you can like go and look at the shop itself.

36. 8:59

    There you go.

37. 9:00

    Yeah.

38. 9:00

    Can you explain CLIP?

39. 9:01

    Oh, yeah. Uh, CLIP is contrastive language to image model trained by OpenAI. It's open source, but it's basically, um, they provide embeddings for text and images. So you can like...

40. 9:17

    A- and the way it works here is that like you embed images and you embed, um, text result. And s- what you can do is you can like do cosine similarity to find the most similar, um, [clears throat] items based on your database for the user query.

41. 9:34

    I don't know if that's clear. Let me know if you have questions.

42. 9:36

    It's the same thing as multimodal embeddings?

43. 9:40

    Um-

44. 9:40

    It's a, it's one way to take image and text to become multimodal. There are different things for audio. CLIP is mostly for connecting text and images.

45. 9:53

    Yeah. Um, I think you can read the paper, CLIP, yeah, um, if you're interested.

46. 10:01

    So I was ke- I was thinking like, "Okay, how could I use Claude in this project to curate relevant recommendations based on the user's requests?" And that's the task.

47. 10:14

    So in a way, you have like user's input. Let's say dress in the style of Emma Chamberlain, blazer like in the Great Gatsby movie, an outfit with a futuristic vibe for the Met Gala.

48. 10:26

    And on the other side, you have like image to text database, um, with images and their labels, and the labels can be produced either by like the original source, uh, or you can use like multimodal model to like come up with labels based on the images.

49. 10:41

    And so the task with the Claude is to curate, like based on the labels from the images, decide whether this like item relevant. Should I recommend this to the user?

50. 10:54

    Is this accurate? Like, does it matches the, uh, user's pers-- like, can I personalize this? Um,

51. 11:04

    and so if you can look at the

52. 11:08

    very simple curation strategy, um, for the prompt. You can just like zero shot it. Like,

53. 11:16

    I need you to decide whether the item is relevant to the user query. Here's the user query. Um, here's the item description. Um, is that item relevant or should, should be recommended to the user based on the user's query?

54. 11:31

    Answer yes or no. Please write the answer in answer tags. Let's unpack this. First of all, Claude really likes XML tags. Really loves XML tags. I think like this is like number one mis-- not mistake, but like, um, one thing that people miss.

55. 11:52

    They don't like put anything in XML tags, and so they don't have like very high like good performance. So everything... Like yeah, if you love XML tags, then you should like put everything in XML tags. [laughs]

56. 12:02

    And, um, with XML tags, you should like be consistent. So what is user query item? And you can be very descriptive. I can share like more examples in a few, uh, later on.

57. 12:14

    And here, in a way you like... The way you interact with Claude is, like, you can see the language here. I need you to decide whether the item is relevant.

58. 12:23

    It almost feels like you talk to a human.

59. 12:28

    Um.

60. 12:29

    Question now.

61. 12:29

    Yeah.

62. 12:30

    So, uh, this XML thing recently only came out, like, maybe like a month or two ago from official Anthropic device.

63. 12:38

    Oh, really?

64. 12:38

    Is this something that was on, like, intentionally trained for or you discovered it after pre-training?

65. 12:45

    I think it was, uh... I mean, we tried to... XML, XML formatting was the first formatting that we, like, kinda like fine-tuned on. Like, um, later on, we discovered, you know, customers need, like, markdown or, like, nee- Claude needs to, like, use JSON, JSON formatting.

66. 13:06

    So like, we kinda, like, learned from customers, but originally it was, like, XML formatting. Yeah.

67. 13:11

    So is that mostly because of the fine-tuning or is that because you had a training set that had tons of XML stuff in it?

68. 13:18

    Uh, I think it's kinda both, yeah. Um, yeah.

69. 13:22

    Do you not need to close your XML tags?

70. 13:24

    Hmm?

71. 13:25

    You don't need to close your XML tags?

72. 13:28

    Close XML tags? [laughing] Yeah, I don't know. Um.

73. 13:37

    Does it, does it generate the HTML?

74. 13:39

    Oh, yeah, sorry. I had a mistake. Yes, I have. [laughing] Um, yeah. And so, like, one, one good thing about XML tags is, like, it's really, uh, easy to extract, right?

75. 13:50

    Like, the strings inside it. And Claude is pretty good at, like, um...

76. 13:57

    I can, I can, like, say, like, "Don't put anything in XML tags." So sometimes, like, Claude will, like, say, "Here's information," blah, blah, blah, blah. But then if you ask, "Just, like, write the answer in, like, this tags," Claude will not put any additional information, which is one of the most annoying thing with, like, language models.

77. 14:14

    So here is the results, uh, that I put. Um, this is through Claude.ai interface. Um, and so yeah, you can see, like, is this item relevant? Uh, says no.

78. 14:27

    Is this item relevant? Yes. But I don't think that's, like, a hundred percent, like,

79. 14:34

    um, you know, perfect system. So it's very, like, zero-shot. So basically, you can, like, iterate, and we will try to iterate more on this.

80. 14:45

    Um, strategy number two is that, like, when, uh, you put-- you ask the model to take some time to think whether the item is relevant or not, uh, in thoughts tags based on the criteria above.

81. 14:58

    Um, and you kinda let, let the model think, um, a little bit more, um, with its reasoning. And then, uh, this is, like, basically chain of thoughts.

82. 15:11

    You can also add, like, criteria. Uh, so as a part of your critique, consider the following criteria. And so if you wanna, like, steer the model on, like, does the item match the specific attributes requested by user?

83. 15:24

    Like, help the model to, like, think kinda, kinda like think through, like, what means... What does it mean to, like, recommend an item to the user? Does the item match the season or were the conditions matching in the user's query?

84. 15:37

    For example, you should not recommend winter coats during summer seasons. So, like, in the criteria, you can, like, give more examples, more elaborate examples.

85. 15:48

    Um, another thing that you iterate on is, like, not just, like, give answer yes or no, but you can, like, based on your critique, score whether the item should be recommended or not, where one is least to be recommended and ten is highly recommended, and put the final score in score tags.

86. 16:08

    And so how does it work? Uh, so, um, here, like, user query James Bond blazer, um, item which I took from, um,

87. 16:19

    I think it was some brand. Um, and Claude would, like, start, like, thoughts tags. Um, overall it seems, like, very relevant, and the final score is nine.

88. 16:33

    And here's another example. I want a dress in the style of The Great Gatsby movie. Um, here's the item, braided cord cropped waist coast. And the critique is basically, uh, the item is not appropriate for the user's needs based on the context clues in the query.

89. 16:49

    Um, it doesn't, like, you know, match the attributes of The Great Gatsby movie. It, like, tries to, like, have, like, some reasoning. And so the score is two.

90. 16:58

    And you can be a little bit more elaborate. This is, like, very simple, like, iteration, um, on, on that. Do you guys have any questions? Yeah.

91. 17:07

    So one interesting thing that I saw, I mean, there was the XML tags-

92. 17:10

    Yeah

93. 17:10

    ... now here that weren't closed. My friend, he is not a native English speaker. His prompts are always in, in very kind of funny English, but he structures them really well and they work really well despite the English being-

94. 17:24

    Mm-hmm

95. 17:25

    ... very incorrect, right?

96. 17:26

    Right.

97. 17:26

    Why? Why does that work?

98. 17:28

    I think it's just the models are, like, pretty good at, like, knowledge transfer between, like, languages or, like, can infer very well on, like, the user's intent. Um, yeah.

99. 17:42

    I don't have, like, pretty clear answer.

100. 17:43

     It's all probabilistic, so that text with one syntax mistake looks close enough to the text with the right syntax. So the probability of the real answer is close in both cases.

101. 17:54

     Right.

102. 17:54

     Like, yes, I don't know.

103. 17:56

     Yeah. Yeah.

104. 17:57

     In this particular example-

105. 17:58

     Mm-hmm

106. 17:58

     ... I'm curious on whether you see any bias with the score.

107. 18:02

     Mm-hmm.

108. 18:03

     In other words, if you were to look at the distribution of scores-

109. 18:05

     Right

110. 18:05

     ... would it be a normal distribution?

111. 18:07

     Yeah. This is an interesting question. Like, this is one question that we ask in our research settings. Like, one thing that we're trying to understand ... Like, we have a research group, uh, called societal impacts, and one thing, uh, that we, we are trying to understand now is, like, when you summarize, like, news articles and you try

112. 18:27

     to evaluate, like, the bias with kind of the distribution. And I feel like this is, like, research active. Like, yeah, I think it depends on the task, um, really.

113. 18:37

     I did not test on this [laughs] literally versus yesterday, uh, prompting. [laughs]

114. 18:45

     Cool. Um, the second task, um ... So Claude is known for 100K co- context, uh, size, which is the entire book of The Great Gatsby can, like, put into the context and you can, like, ask the model, uh, summarize the book or, like, uh, ask some tasks based on the huge context.

115. 19:07

     And this is, like, m- basically time test compute, um, thing. And so with long context, um,

116. 19:19

     let me see. The way you can use long context can be in different ways. Like, one way is, like, you put multiple documents and try to summarize or, like, uh, retrieve information based on the documents.

117. 19:33

     Another way to use long context is to have a bunch ... a huge few shot prompt.

118. 19:40

     And so as you know, like, chain of thought, um, technique relies on the stated reasoning faithfully reflecting the model's actual reasoning. And in one of the recent papers, we found that it's not super ...

119. 19:54

     or, like, it's not always the case. So

120. 20:01

     doesn't ... So basically what it means is that, like, if you ask the model do a chain of thought, it might not necessarily attempt to, you know, chain of thought, uh, to produce the final answer.

121. 20:14

     It might just, like, ignore it or, like, uh, not take any account. So it's not ... We, we call it, like, unfaithful basically. It's not super faithful.

122. 20:25

     And so we propose in this paper, um, like, decomposition-based methods can actually achieve, like, strong performance on specifically question answering tasks, sometimes approaching that of chain of thought performance while improving the faithfulness.

123. 20:45

     Do you guys have any questions?

124. 20:47

     I didn't understand that. Sorry.

125. 20:48

     Okay. Yeah.

126. 20:49

     What is faithfulness here? Is the faithfulness to your prompt or to your-

127. 20:54

     Um, faithfulness is, um ... Yes, to your prompt. Yeah.

128. 21:01

     What's decomposition? What's decomposition again?

129. 21:03

     Yeah. Uh, let me explain what decomposition is. So here's the graph from the paper. Uh, we have, like, three methods. First is a chain of thought method, which is like, uh, here's the question: Could Scooby-Doo fit in a kangaroo pouch?

130. 21:20

     Uh, there are two choices: A, yes; B, no.

131. 21:24

     Chain of thought prompt saying, like, [clears throat] "Let's think step by step," gives the reasoning. Um, the human ask the follow-up questions based on the above, what is the single most likely answer choice?

132. 21:39

     And, and the model says the correct answer choice is B,

133. 21:43

     right? The chain of thought decomposition is when you decompose a question ... when you can ask the model to decompose a question into, like, multiple sub-questions so that each sub-question are kind of independent from each other.

134. 21:58

     Because in chain of thought, like, you have one, two, three, you know, like, they kind of, like, can influence each other, right? Like, in decomposition you kind of like, um, y- y- you decompose and you, like, put each s- sub-question in the independent context.

135. 22:14

     So in a way, it kind of, like, reduces the bias.

136. 22:19

     Um, and so in this, um ... Uh, let's, let's see here is, like, sub-question one: Uh, what type of animal was Scooby-Doo? The answer from the model, Scooby-Doo is a fictional character.

137. 22:32

     Another sub-question f- for, uh, for the assistant for Claude, how big is an average kangaroo pouch? And you ... And what you can see is that, like, each sub-question is kind of, like, self-contained.

138. 22:43

     It's very atomic self-contained question. Um, and so you have, like, multiple sub-questions like this, and then what you do is you recompose. So, like, you, like, put sub-question, answer sub-question, answer sub-question, answer into, like, one context and ask the model, based on the above what is the single most likely answer choice?

139. 23:05

     The correct answer choice is B. Yeah.

140. 23:08

     Um, i- in chain of thought, like, in the system prompt or whatever the user's prompt is, like, we mention let's think step by step.

141. 23:15

     Mm-hmm.

142. 23:16

     Uh, w- what do you do for the decomposition? Is there, like, a similar,

143. 23:22

     you know, input to the model-

144. 23:24

     Mm-hmm

145. 23:24

     ... to make it decompose into multiple questions?

146. 23:26

     Yeah. Uh, I can share the prompt, um, in a few slides, uh, on this. Um, but, uh ... Yeah. Any other questions?

147. 23:35

     Can you show the graph again? Now I know it is-

148. 23:37

     The, this graph?

149. 23:38

     Yeah.

150. 23:49

     Um, yeah. Let's look at the prompt. Um, very hard to see, but I'll share the slides. Um, let's ... Um, I, I'm, I'm gonna give you, like, legal context, like, legal question.

151. 24:06

     Let's say you have a question on, like-[clears throat]

152. 24:09

     A, a legal question and you ask us like, whi- which of the following is the most persuasive argument that a person is liable to the creditor under the terms of the agreement and here's the context?

153. 24:20

     So that's the question, basically. And so you have like choices for the model. So this is like multiple choice question and before that you have like a huge few shot prompt.

154. 24:33

     Um, and basically here, uh, to answer your question, like it says, "I'm going to give you a question. I want you to compose into a series of sub-questions. Each sub-question should be self-contained with all the information necessary.

155. 24:48

     Um, this is really important, blah, blah, blah. Uh, make sure not to decompose more than necessary. Um,

156. 24:59

     be concise, blah, blah. Please put each sub-question in like these tags but include the numbers corresponding to each tag."

157. 25:08

     So, um, and the model says, "Yes, I understand. Uh, you have a question,

158. 25:16

     um, multiple choice answers," and the model provides sub-questions for you and then what you do is that you try to answer the first sub-question and you give it to the model.

159. 25:29

     You try to answer the second sub-question, you give to the model. Third sub-question, you give to the model and then later you say like based on everything above like you give all the context, um, answer me the question.

160. 25:44

     Um, the correct answer C. Yeah. And so this is like very similar in the legal context. You have sub-questions like what is consideration in contract law?

161. 25:59

     Blah, blah, blah. And you have-- you can have like another model to like sample here. You can have like another model to answer this. It doesn't necessarily should be like one model.

162. 26:08

     Um, and then there's like another sub-question and here's the answer.

163. 26:17

     Yeah. Do you guys have any questions? Um,

164. 26:26

     the second thing that I wanna talk about is how to use Claude to do evaluations. Like evaluating like Claude on like long context ability. Let's say you have a lot of like documents and you want to understand how good Claude is answering questions based on the document

165. 26:51

     or is it, uh, is it able to answer like the questions not just like from its pre-trained knowledge but like based on the document itself.

166. 27:02

     And so, um, I'm gonna give you example that we did at Anthropic. Um, multiple choice QA, um, evaluation design. So our goal was to-- with this exercise to evaluate techniques to maximize Claude's chance to correctly recalling a specific piece of information from a long document and so the document that we chose was a government document

167. 27:28

     that contains like a bunch of like meeting transcripts, different departments and we also chose the one that was like, uh, from this year July thirteens, uh, which is like way after Claude's, um, training data cut off so that you don't like, um, you have the document that does not have in the pre-trained knowledge or something.

168. 27:53

     And so what you're trying to do is like now you want to use Claude to generate question-answer spares. Um,

169. 28:05

     you, you in a way like you create like data, data set based. You use language models to create like data sets and so the way you do that is that you split the document into sections and use Claude to generate like five multiple choice questions for each section.

170. 28:21

     Each with three wrong answers and one right answer and if you do that you then reassemble like randomized sets of those sections into like long documents that you could pass them to Claude and test its recall, uh, of their contents.

171. 28:38

     This is very meta. Let me know if you have questions. Yeah.

172. 28:42

     Makes sense.

173. 28:44

     Um, so here's a prompt to generate multiple choice questions. Um, I a- we ask like please write five factual questions for this. Um, [clears throat] some guidelines at the end.

174. 29:01

     Um, and basically we test different strategies, prompting strategies just asking Claude, give Claude two fixed examples of correctly answered general knowledge and, um, that are unrelated to the government document.

175. 29:18

     Um, providing two examples and providing five examples of correctly answered questions and we tested the strategies, uh, on different settings like one is containing the answer positioned at the beginning, the end or the middle in the input and we tested with like seventy k and ninety-five k token documents.

176. 29:43

     You can look at the prompt and more specific how we did this in our blog post

177. 29:51

     but basically the result is this. Uh, here we see that um-

178. 30:02

     What is the metric?

179. 30:05

     Yeah. The metric was um, to... Let's see

180. 30:15

     Like, basically, how many, how many times, like, Claude has correctly answered the question. Um, right. And so...

181. 30:31

     Yeah, sorry. Uh, basically what we find is that, like, for document Q&A, asking the question at the end of the prompt performs a lot better than asking at the beginning.

182. 30:42

     You can see it here. Uh, pulling relevant quotes into, like, critique or, like, thoughts tags is helpful. Um, it's like the small cost to latency but improves accuracy.

183. 30:58

     Uh, and we tested on, like, both Claude and Claude Instant.

184. 31:03

     Um, and it seems like you can boost way better performance from, like, Claude Instant, um, than Claude 2.

185. 31:15

     Basically, the idea is that, like, if you wanna use long doc Q&A, put the instructions at the end of your prompt. Yeah. That's, like, the result of this. [laughs]

186. 31:27

     Wait, what was the-- I didn't catch, what was the scratchpad in the-

187. 31:32

     Uh, oh, yeah. Like, you just ask the model to, like, put thoughts in, like, thoughts tags before answering the question, so it has, like, more reasoning-based approach. Yeah.

188. 31:42

     Can you go to the table again? Sorry, I just wanna-

189. 31:44

     Yeah. Sorry. I'm, uh... [laughs]

190. 31:47

     So just for the averages for me. But, like, the outcome was basically putting it at the end matters more than all the other-

191. 32:02

     Yeah

192. 32:03

     ... optimizing strategies.

193. 32:04

     Right. Yeah. Um, yeah. In a way, this is, like, an example to show, like, how to use Claude to generate a dataset that you can, like, evaluate and, like, you can use it for, like, evaluation, basically.

194. 32:27

     Yeah.

195. 32:28

     So this has to do with you putting your instruction at the end of the prompt.

196. 32:32

     Mm-hmm.

197. 32:32

     Uh, first are there theories on, like, why, why specifically instruction should be at the end?

198. 32:37

     Mm-hmm.

199. 32:37

     And are there any things like... Do we have any understanding of,

200. 32:43

     like, are there certain things at the beginning of the prompt that still might be weighted or is it, like, this sliding scale that, like, the further in the beginning of the prompt, like, the less attention it gets or influence it has?

201. 32:53

     Yeah. I think that's basically the hypothesis.

202. 32:55

     Okay.

203. 32:55

     It's like the, you know, it, it's like the distance. It's like the model attends more to the end of the prompt-

204. 33:03

     Okay

205. 33:03

     ... uh, than the beginning.

206. 33:04

     It's not like people were... There's exact [laughs]

207. 33:06

     I think there was a paper saying, like, it just forgets in the middle or something.

208. 33:09

     Oh, okay. Yeah.

209. 33:10

     Um, yeah, I think this is the problem with, like, long context that we're trying to fix or something. Yeah.

210. 33:16

     So to, to follow on that question, you are saying that there was a paper that said that it remembers the beginning and end and kind of forgets in the middle.

211. 33:24

     Yeah.

212. 33:24

     But what you're saying is for Claude 2 it seems to do best if you give it at the end.

213. 33:29

     Yeah.

214. 33:30

     So that paper doesn't apply to Claude?

215. 33:32

     Um, I did not read that paper like-

216. 33:37

     No, I'm just curious. Like, what you're saying is you're finding it, at least for Claude-

217. 33:42

     Right

218. 33:42

     ... the end part gets more attention.

219. 33:45

     Yeah. For, like, a specific task is, like, a long context, uh, Q&A for, like, long documents. Yeah. Um, yeah, we have not tested on other tasks to my knowledge.

220. 33:59

     Um, so the prompts you showed were using regular prose and then the XML tags.

221. 34:05

     Mm-hmm.

222. 34:06

     Um, I think that's also what's in the Anthropic docs. Have you guys ever done experiments on, like, that kind of format versus markdown-

223. 34:16

     Mm

224. 34:16

     ... versus everything is in XML? Do you have any thoughts on that?

225. 34:20

     Yeah. So, um, in general I think, I think it's because

226. 34:30

     markdown was kinda like... There's not that much of like...

227. 34:37

     I don't know. It's, like, best in XML tags. Like, I'm, I'm, like, thinking the-- I've, like, tried Claude to like, you know,

228. 34:45

     use, like, JSONL or, like, uh, use markdown, but sometimes it's like, you know, it's not as good as, like, XML. With XML it's almost a hundred percent accuracy. Yeah.

229. 35:03

     Let's see. Yeah. Um, let's go to another task, um, which is, like, you can use language models to, like, auto-label basically anything. Um, so one of the examples that we did last year, um, we asked Claude to categorize the labels for the clusters.

230. 35:28

     And so, um, this was for the paper but the approach was very simple. We have a bunch of like, you know, texts and we embed them in UMAP. Um, and we do, like, KNN clustering

231. 35:46

     and for each cluster... Sorry, K-means clustering. And for each cluster select, like,

232. 35:56

     for each cluster aggregate all the, you know, little, like, statements, uh, claims and we ask the model to come up with a category for this cluster.

233. 36:08

     So that's the approach and you can look at the other

234. 36:15

     labels Uh, labels here are not super good because we used Claude 1.3 at that time. Claude 2's supposed to be, like, way better at this. Um, but this is, like, you know, cached.

235. 36:27

     It was, like, last year. Um, where's my slides?

236. 36:35

     And so one thing that you can do with this kind of task,

237. 36:45

     we call it self-consistency. You can generate N samples for the question. Um, so let's say you have a question like, "How do you label this cluster?" And you generate independently N times, and you can ask just, like, come up with, like, one category.

238. 37:07

     Um, well, this method is mostly useful for, like, quantitative, like, if you have, like, a math question and you sample, like, different... like, sample multiple times and come up with the answer.

239. 37:19

     Um, like, the most common answer is, uh, the one that you select for the final answer, and this is called the majority vote.

240. 37:29

     Another technique that you can use is, like, um, have, like, two generated samples and ask another model to evaluate whether those samples are consistent or not. And if the samples are consistent, the-- well, you gain more confidence that this is correct, right?

241. 37:44

     And if it's not consistent, you just, like, deselect.

242. 37:50

     Um, another thing that you wanna do with Claude is, uh, if you--

243. 37:56

     If Claude is kinda, like, misses the nuance, especially for, like, categorizing a lot of labels and you have, like, a, a lot of, like, categorizations, um, [clears throat]

244. 38:08

     you can add contrasting conceptual distinctions in your instruction, and you can do it in multiple ways. One way to do it is, like, you provide bad example. Let's say, like, "Here is a very bad category, and you should never come up with it because this is, like, too narrow or, like, too general, and this is not what

245. 38:27

     I want." Uh, like, give, like, contrasting, like, examples. Uh, vary the context. Use examples in different contexts and settings. Um, not just, like... Just, like, have, like, more diversity.

246. 38:40

     Like, diversity is, like, um, an... Uh, uh, the more diverse, like, few-shot prompt examples, the, the better.

247. 38:47

     Use analogies and metaphors. Um, if the concept is, like, too hard to understand for the model, try to, like, decompose and, like, bring analogy. Um, point out, like, common misconceptions, um, especially for, like, categorizing, like, [clears throat]

248. 39:05

     let's say, what is false presupposition, right? Like, uh, point out the common misco-misconception and, like, clarify, like, why this is, like, incorrect. Like, provide examples, like, specifically show why common mis-misconception is wrong.

249. 39:26

     Uh, yes. Do you guys have any questions? Yep.

250. 39:31

     So for the, the label, the goal is you, you run K-means to get groups and then you wanna give a name to the group. Is that the label matching?

251. 39:41

     Just to clarify.

252. 39:43

     Um, I cannot super hear, but, um-

253. 39:45

     Yeah. It... So the goal here is you're, you're running, uh, a clustering algorithm to get groups.

254. 39:50

     Mm-hmm.

255. 39:51

     And then the, the task of the LLM is to put a name on each group?

256. 39:55

     Yeah.

257. 39:57

     Okay. Okay.

258. 39:57

     Like, come up with a category. Like, um, yeah, come up with a category or, like, classify, uh, [clears throat] like, label that cluster basically.

259. 40:09

     Um, so here is, like, ba-very basic, like, tips and strategies with Claude API. Um, number one is formatting. Um,

260. 40:18

     like, Human: Assistant is, like, what Claude loves, and if you misses, you miss it, like, you'll get, like, very, very terrible results. Uh, new line, new line, Human, new line, new line, Assistant.

261. 40:31

     Um, yeah. Uh, you can also put words in Claude's mouth to, like,

262. 40:39

     kinda, like, say, like, "Do you understand it?" And you can, like, put in the, um, Claude's mouth, "Yes, I understand it," in a way to, like, you know, put, put the model i-into these modes.

263. 40:54

     Have Claude repeat instructions back. Um, you can say, like, "Do you understand the instructions?" Um, and you can put, like, Assistant, "Yes, I understand instructions,"

264. 41:09

     blah, blah, blah. Uh, to reduce hallucinations, like, let Claude

265. 41:17

     hedge and, like, say, like, "I don't know," or, like, uh, [laughing] "I don't have enough information or, like, context to answer the question."

266. 41:29

     Um, here is another thing. Um, if you have, like, generate direct quotes, if you have, like, a document or, like, um,

267. 41:40

     a long document in the context, um, make Claude to say, "Find appropriate quotes," but also say, like, um, "If there are no quotes in this document that seems relevant to this question, please just say, 'I don't find any relevant quotes,'" so that it doesn't make up

268. 42:00

     or fabricate new quotes. Uh, how to give good examples. Um, are the examples similar to the ones you need to classify? Are the examples diverse enough for Claude not to overfit to, to the specifics?

269. 42:16

     Equally distributed among answer types. Don't always choose option A, but, like, y- kinda, like, have the diversity.

270. 42:25

     Um, yeah. I get a lot of quest-

271. 42:30

     Oh.

272. 42:30

     Yeah.

273. 42:31

     In the first example you said the formatting was the, the bad part. I can't see exactly what's wrong with the formatting in that.

274. 42:39

     Oh, formatting in a way... Oh, here?

275. 42:42

     Yeah.

276. 42:43

     Um, I think they didn't put like new line, new line. Pretty sure.

277. 42:54

     Oh, the... Here or here. Sorry. Here, you put like human assistant inside the XML tags. You only need-- You only have to use human assistant

278. 43:05

     as like tokens to like sample, but you should never put it in like, like inside the context itself. Either use like user and like other like, um, you know, um,

279. 43:18

     words like user AI or something, or like H or A, but you should never use human assistant. Human assistant is like very special, special words.

280. 43:29

     If, if that didn't have the XML tags, would it, would it be okay?

281. 43:33

     Um, it would be okay, but you would like make... You should like have human and then assistant in between, and then human assistant and then another human and assistant, basically.

282. 43:48

     Uh, yeah, formatting is like human assistant, human assistant. You should never have like human, human, assistant, assistant or something. Uh, that's bad. Um, yeah.

283. 44:01

     I think we have like more extensive, uh, explanations in the API docs, if you can look at it.

284. 44:10

     Yeah, I get a lot of questions like what the future of prompt engineering is. Um, and I think the answers are pretty clear. Like prompting will stay. We'll just ask like more complicated, nuanced questions or like tasks for the model.

285. 44:25

     Um, prompt engineering is a, we will like-- We're moving towards the world where we'll have like more and more synthetic data generation. And so I'm pretty optimistic about like mo- using models to like generate like diverse sets of like datasets.

286. 44:40

     Um, you can also use language models to write like evaluations. Um, so you use prompting to do that. Um, reinforcement learning from AI feedback, um, is an alternative to like reinforcement human-- from human feedback, which is like a little more scalable.

287. 45:01

     But basically, you ask the model to revise its own, uh, responses in the process. So you give the model-- Y-you like ask the model to like self-reflect or like self-revise.

288. 45:13

     Um, and so you use prompting in that process to do this.

289. 45:18

     And especially like prompt engineering will become like a, a standard part of like product development. I feel like, um, things that we did in Claude products such as like auto-generating titles.

290. 45:33

     Like these things was never like done before, like cl- cl- uh, before like large language models. And so you can like create delightful mini UX experiences, uh, such as like that using just prompting or something.

291. 45:48

     And you can like have personalization. Uh, maybe you can embed all the users conversations and like suggest like new topics for the conversation. Um, and you can use models to do that.

292. 46:03

     Um, and the most like interesting thing is like, uh, finding most optimal prompts for specific tasks. Maybe you want to like minimize the number of tokens to get the highest accuracy for the task.

293. 46:13

     Um, yeah. Uh, here are some resources. Uh, we just, uh, launched Anthropic Cookbook with like certain like demos on research, uh, on retrieval and search. Um, we have prompt design guide w-- in API, uh, book.

294. 46:30

     Um, you can also read our, the papers that we publish. Uh, oftentimes we have like appendix with like all the prompting that we do.

295. 46:39

     Yeah. Thank you so much. And, uh, if you have any questions, let me know. [audience clapping]

296. 46:50

     Uh, yeah, we have five minutes for questions. Charles is coming up. We also have water. Thanks to Sean, um, for bringing in some water so, uh, get hydrated people.

297. 46:59

     But five minutes for questions for anything about prompt, large prompt engineering.

298. 47:08

     Yep.

299. 47:09

     So have you tried these with different things besides Claude, like other...

300. 47:14

     Have you had similar kind of results? 'Cause you're talking about Claude in this one particularly. Like maybe like with OpenAI or other-

301. 47:24

     Yeah. I think I'm most, um, experienced with Claude because I use it like every day. Um,

302. 47:32

     less experience with GPT. Uh, I did not look carefully, to be honest, at their like API docs. Um, but it seems like the strategy is a little bit different.

303. 47:44

     Yeah.

304. 47:45

     Okay.

305. 47:45

     They don't have like formatting as VR, let's say. Yeah.

306. 47:49

     Uh, great, great presentation and, and that work was very impressive.

307. 47:54

     Thanks.

308. 47:54

     Uh, have you seen this prompt reader that came out recently? Is that, uh, where you basically using LLM to improve the prompt, uh, considering the outcome? Uh, do you see that in the-- inside the future, say, optimal tasks?

309. 48:06

     Is that what you mean or do you see other directions for, uh, optimizing prompts?

310. 48:12

     Yeah. I think, uh, that's, that's actually one of the directions too. Like, um, rem-- Don't remember

311. 48:20

     how, what was that paper called? Like LLMs as like optimizers, I think, right? Um,

312. 48:26

     but yeah, I guess like, um- In a way, there are, like, certain tasks that the models are, like, not good at currently. Like, for example, like, self-correction. Like, the models are not really good at, like, self-correcting their, like, answers.

313. 48:40

     And, like, can you find, like, a prompt that is, like, pretty good at it or, like, um, other tasks that you want? Yeah.

314. 48:51

     So I'm curious-

315. 48:52

     Yeah

316. 48:52

     ... about what techniques your team is using for actually, like, evaluating the quality of the responses. That's something that a lot of teams kind of struggle. Like, how do we actually automate this?

317. 49:01

     Yeah. I think, um, depends on the task. Sometimes we just, like, have to look manually, qualitatively, uh, at outputs. Um, sometimes you, let's say, um,

318. 49:14

     you wanna evaluate, you know, how much does the model refuses and if it refuses in a relevant context or not. And so, uh, you use, you know, generated answers, and you categorize refusals [laughs] in different categories and use the model to categorize that.

319. 49:35

     And so you just, like, see the rate, um, yeah.

320. 49:40

     I can think of that example. Yeah. It depends on the task. Some tasks are, like, you know, um,

321. 49:48

     for, like, hallucinations, you ha- actually have to, like, look yourself or something. Yeah.

322. 49:53

     Yeah.

323. 49:54

     So you mentioned, uh, something about clustering, uh, different kinds of questions that we need to ask different categories and, uh, giving it a label. Uh, so have you tried using OpenAI functions which has, uh, some enumeration limited times with the predefined classes or intents, user intents and have you seen if that improves the accuracy of the classification

324. 50:18

     when you try to-

325. 50:20

     Of with OpenAI?

326. 50:22

     Of, of any kind of function calls. Uh, the OpenAI function call is a model.

327. 50:26

     Uh, if it's any kind of, uh, model that takes structured data that can, uh, run further model functions.

328. 50:34

     Yeah. Yeah. I think, uh, um, I wouldn't say m- too much about this, but I actually didn't-- have not, like, extensively used function calling from OpenAI or, like, other models.

329. 50:45

     It limits the, the classification if you don't-

330. 50:48

     Right

331. 50:49

     ... use it.

332. 50:51

     Yeah. Cool. Yeah. Yes.

333. 50:54

     So you mentioned something about, you know, using an LLM to generate titles for customers. How did you actually try to evaluate if the titles are relevant-

334. 51:04

     Mm-hmm

335. 51:04

     ... and actually consistent, like-

336. 51:06

     Mm-hmm

337. 51:06

     ... beyond human feedback?

338. 51:09

     Yeah. I think, uh, one int- Uh, actually this is an interesting question. Like, I worked on the auto-generating titles for Claude.ai, and, um, one thing that I asked Claude is to be like an editor, like, have an editorial taste.

339. 51:25

     And what we did is actually we took previous titles, and we put in the context to generate a new title. And so in a way, it's, like, a little more consistent to, um, what the style of the user is.

340. 51:41

     Yeah.

341. 51:41

     Would you be able to share the recall precision in terms of catching defects if they do happen or-

342. 51:47

     Uh, yeah. I'm not sure if I can share that. Uh, yeah.

343. 51:51

     Okay. Was it reliable enough to use in production?

344. 51:54

     Uh, yeah. We, we use this in production. I can, like, show you, like, Claude.ai interface.

345. 52:00

     And, uh, one thing that we changed recently is that, like, if you have, like, pretty, like, short, like, um,

346. 52:08

     you know ... Sometimes you don't have-- you don't need, like, LLM to come up with a title. You just take... If, if the prompt is, like, very short, you just, like, um, use the, like, the first, like, words.

347. 52:19

     Um, but here, yeah, like, I don't know. Um,

348. 52:23

     let's see. Here is introduction, but then, like,

349. 52:29

     recommend some books. Um, yeah, I don't know. Uh, [laughs] yeah.

350. 52:40

     Uh, I have a quick question.

351. 52:41

     Mm-hmm.

352. 52:41

     Did you cover the difference between Claude and Claude Instant?

353. 52:45

     Um, no, but I can tell. Uh, so Claude-

354. 52:51

     For those who don't know, she trained Claude Instant. And I'm like, "You might as well cover the thing that you did." [laughs]

355. 52:56

     Yeah. [laughs] [laughs] So let's look at the ... Is there some docs on this? [laughs]

356. 53:02

     When did you guys announce the 1.2 thing?

357. 53:05

     When did we announce? Uh-

358. 53:10

     August 9th

359. 53:11

     ... August 9th? August 9th. Um, basically Claude 2 is a larger model. Uh, it's a little bit smarter. It's, like, smarter than Claude Instant. Claude Instant is way cheaper and way faster.

360. 53:25

     But Claude Instant is better than Claude Instant one, um, in the, like, more, like, reasoning-based tasks. So it's b- way better at math. Uh, it's way better at code.

361. 53:36

     Um, other benchmarks are, like, pretty similar, but I think we specifically trained Claude Instant to be good at, like, math and code.

362. 53:47

     Um, and it's, uh, way better at, like, red teaming. Um, like, automated red teaming evaluations, so it's more robust to, like, jailbreaks.

363. 53:57

     Um, yeah. I really like this model. You guys should use it.

364. 54:03

     Good. [laughs]

365. 54:03

     Yeah.

366. 54:05

     Thank you.

367. 54:05

     Yes.

368. 54:05

     We exposed, exposed my ignorance. So when you talk about training, like, when you train this Claude Instant, was that ta- fine-tuning or was that something different?

369. 54:13

     Uh, yeah, fine-tuning.

370. 54:15

     Okay.

371. 54:15

     Yes.

372. 54:15

     Last question.

373. 54:19

     Can, can you say more, I guess, red ... Like, what is red teaming and how does that work?

374. 54:23

     Yeah. Red teaming is interesting concept. It's, um, basically you ... Like, the models are pretty, like,

375. 54:32

     uh, vulnerable to, like, certain, like, jailbreaks. Um, so sometimes, let's say, like, a very simple example. Like, can the model give you instructions how to build a bomb? And so we, we consider it as a jailbreak.

376. 54:46

     And so the goal is to, like, uh, in that cases, like, the model should, like, refuse or do not, like, provide any additional information in case of, like, um, unsafe, like, prompts or something like this.

377. 55:00

     And so this is, like, the internal evaluations that we have. Um, you can read in the model card that we ha- we launched in Claude 2 how we specifically do that.

378. 55:11

     Uh, but it's basically the amount of, like, um, how robust the model is to, like, those jailbreaks. Yeah.

379. 55:20

     Cool.

380. 55:21

     Thank you. Thank you very much.

381. 55:22

     Yeah. Thank you. [audience applauding]

382. 55:28

     Yeah. Uh, so-