AI Engineer Summit 2023
No-code Fine-tuning: Mark Hennings
Read the talk
No-code fine-tuning: from examples to a press release writer
Mark Hennings shows how a team can turn examples into a specialized language model, evaluate it against a prompt, and build a press release workflow around facts.
From a talk by Mark Hennings
Before you start: Familiarity with prompting a language model is helpful; no programming or model-training experience is required.
What fine-tuning specializes
How do you write a product description, normalize messy text, or decide which support issue deserves attention first? These are recurring business tasks, but specifying the right behavior as rules can be difficult. Fine-tuning means training a foundation model for a specialized task. Mark Hennings opens with three overlapping groups:
- Writing: emails, blog articles and product descriptions.
- Transforming data: scrubbing fake emails from a list, extracting or normalizing information, translating, paraphrasing and rewriting.
- Making judgments: qualifying sales leads, ranking support priorities, detecting fraud and flagging inappropriate content.
The common challenge is interpreting language with enough nuance to produce a useful result. Hennings argues that language models handle this more easily than traditional programming or rule-based approaches.
A prompt can already accomplish many of these tasks. The question is what changes when recurring instructions and examples become training data for a specialized model.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Speed, cost and shorter prompts
The proposed efficiency gain starts with matching the prompted model’s quality using a lighter model. If training teaches that smaller model the required behavior, inference can become faster and cheaper. Removing repeated instructions also shortens inputs, leaving more context space for completions. Training examples provide another place to express edge cases and let teammates contribute improvements.
Hennings also asserts that fine-tuning is naturally resistant to prompt injection, but he does not demonstrate an attack or a defense. Ordinary task fine-tuning should not be treated as a security guarantee.
For speed, Hennings compares GPT-4 with GPT-3.5 on the OpenAI API, then gives a cost example comparing GPT-4 with fine-tuned GPT-3.5. These are figures from the historical 2023 demonstration, not current model or pricing guidance.
| Comparison | Reported result |
|---|---|
| OpenAI API response time per token | Hennings reports approximately 196 ms for GPT-4 and 73 ms for GPT-3.5. |
| GPT-4 versus fine-tuned GPT-3.5 cost | Hennings reports 88.6% savings in his example. |
He describes the latency difference as roughly three times faster. The talk does not specify the model snapshots, workload or measurement procedure, and it does not supply the token counts or training-cost allocation needed to reproduce the savings calculation.
The prompt-compression example is a blog writer. Its engineered prompt specifies the topic, writing approach, tone, desired behavior and things to avoid. With fine-tuning, examples teach the recurring writing behavior; the request can concentrate on the topic that changes from one article to the next.
Hennings describes this particular blog-writing prompt as 90% shorter. The reduction depends on how much of the original prompt consists of reusable instructions. The useful separation is between behavior shared across requests and information unique to the current request.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Training data as a shared working surface
Imagine a GitHub repository whose entire codebase lives in one file. Hennings uses that as an analogy for a team maintaining one enormous prompt: everyone must work on the same artifact. Fine-tuning offers something closer to a repository with multiple files, where contributors can work on different pieces. Those pieces are training examples. Team members can add, edit and improve examples in a shared dataset, which then feeds the fine-tuned model.
This makes the dataset a collaborative layer for specifying behavior. The reason to adopt it remains conditional: the resulting model needs to produce equal or better output. Hennings then turns to the practical obstacle—fine-tuning still looks like a developer’s job.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Lowering the technical and data barriers
A typical fine-tuning tutorial can involve provisioning GPU servers for training and inference, formatting data with ad hoc Python scripts, configuring parameters and making API calls. Hennings’s proposal is to automate that operational work behind a user interface. The person who knows what a good answer looks like should be able to contribute without owning every infrastructure step.
Hennings says 20 examples can be enough to start a fine-tuning experiment. They can even be handwritten, avoiding the initial burden of assembling the thousands of examples he associates with traditional machine learning. Starting an experiment is distinct from establishing production quality; that requires evaluation.
His intuition comes from few-shot learning. A prompt might contain five examples of the desired behavior. A training dataset can instead contain 20 or 100, without placing all those examples into each request. This extends the amount of example-based guidance available to the model. Hennings presents improved task fit from additional examples as an intuition, rather than a measured result.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Prototype, evaluate and learn from production
Prompt engineering remains the starting point in the proposed development lifecycle:
- Prototype with a prompt. Validate the concept and use the prompted model to help create an initial training dataset.
- Fine-tune and evaluate. Check whether the trained model improves on the prompted version, then test which models can achieve the required performance.
- Deploy after evaluation. Put the selected fine-tuned model into production.
- Feed experience back into training. Capture user feedback and log real examples, then incorporate useful cases into the dataset for subsequent improvement.
The loop connects the initial specification to the situations users actually encounter.
Developers can operate this process, but they need not be its only participants. People who understand the task can help with prompting, example creation and fine-tuning. Hennings introduces Entry Point, identifying himself as a co-founder, as tooling intended to make that participation practical.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Building the Press Release Writer dataset
The demonstration opens the Press Release Writer project in Entry Point, containing 20 examples. Hennings began with the desired outputs: he found 20 strong press releases through articles collecting examples of effective releases. That gave him writing to learn from, but not complete input–output pairs.
To supply the missing inputs, he used GPT-4 in ChatGPT to work backward from each release. The task was to extract the list of facts a professional writer would need to produce it. Each resulting pair therefore associates a factual brief with a finished press release.
Facts are the input; polished prose is the output. Hennings chooses that boundary because he does not want to rely on the model for factual knowledge. The intended result is a strong first draft built from supplied information. The examples teach how to turn that information into a release.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
From structured fields to a training job
Entry Point exposes the training data as structured records. When a CSV is imported, each column becomes a field; this project has fields for the facts and the press release. Keeping them visible makes it easier to inspect what will enter the model’s training data.
Fields are inserted into input and output templates using Handlebars, much like inserting a contact’s first name into a personalized email. With illustrative field identifiers facts and press_release, the input template can be:
handlebars
{{facts}}
The corresponding output template can be:
handlebars
{{press_release}}
Applying those templates to each record supplies its factual brief as the input and its existing release as the target output. The templates define the format once; the records provide the changing content.
Fine-tuning does not require abandoning instructions. The demonstrated GPT-3.5 Turbo workflow also supports a system prompt. Hennings uses that to describe a hybrid: examples teach the task while a system instruction provides additional guidance, including when the dataset is small.
Once the dataset and templates are ready, the interface handles the training setup:
- Open
Fine-tunesand selectAdd. - Choose the model and platform; Entry Point supports multiple platforms.
- Review the token count and estimated cost.
- Press
Startto launch the job.
Hennings quotes an estimated training cost of about $1 for the demonstrated job. The talk does not provide the token counts, selected model or training settings needed to reproduce that estimate. He then switches to an already-trained model in Entry Point Playground, rather than waiting for the new job to finish.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Iterating on facts instead of prompt boilerplate
In the Playground, Hennings supplies facts he wrote about the AI Engineer Summit. The trained model generates a draft with a title and recognizable press-release formatting. The demonstration shows the facts-to-draft workflow; it does not establish held-out quality or factual accuracy.
The generated draft becomes material for another pass:
- Write the list of facts.
- Generate the article.
- Read the draft and use it to identify ideas or refinements.
- Revise the facts and generate again.
This is the authoring loop Hennings finds useful. Training data carries recurring writing conventions, so each iteration can focus on what the release should say. The user works on the factual brief while the trained behavior supplies much of the writing boilerplate.
Hennings closes by mentioning Entry Point’s data-synthesis features and tools for comparing fine-tuned models, neither of which is demonstrated here, and directs viewers to the product. The completed walkthrough ends with a more focused writing process: inspect the draft, improve the facts, and let the examples carry the reusable style.
Suggest correction
This note stays in this page until you copy or download it. Nothing is submitted; reloading clears the draft.
Resources
From the talk
The platform demonstrated in the talk, now offering fine-tuning, synthetic-data generation and data transformation workflows.
Official guide to substituting field values into templates, with interactive examples.
Further reading
A 2023 walkthrough of CSV imports, field templates, training, validation and the Entry Point playground.
The original announcement documents narrow-task results, prompt compression and launch-era training and inference prices.
A contemporary explanation of chat-format training examples and combining system instructions with fine-tuning.
Updates since the talk
Later research explains how targeted instruction-hierarchy training improves resistance to prompt injection.
Read the complete timestamped transcript
- 0:00
[upbeat music] Hey, my name is Mark Hennings.
- 0:22
I'm a serial entrepreneur, and I'm super excited to talk to you about fine-tuning large language models today without any code. So let's begin. For our purposes today, fine-tuning is training a foundation model for a specialized task.
- 0:38
Some examples of these specialized tasks are writing any kind of copy, emails, blog articles, product descriptions. It could be scrubbing fake emails from a list, extracting or normalizing data, translating, paraphrasing, rewriting, qualifying a sales lead, ranking priority of support issues, detecting fraud, or flagging inappropriate content.
- 0:59
These are very common tasks that businesses do every day, and something they have in common is that traditional programming or rule-based approaches do not work well for them. But large language models are great at them.
- 1:12
They perform them easily, and they can capture the nuance in the text that you're working with. So why should we fine-tune? I mean, prompt engineering is great, right? You can do almost all of these things with a prompt.
- 1:23
Well, I'll tell you. Fine-tuning is awesome. It's faster and cheaper because you can train a lighter model to match the quality of what you were doing with a prompt.
- 1:33
It reduces the size of your prompts, allowing for longer completions. Training examples allow you to cover edge cases and collaborate better as a team,
- 1:42
and it's naturally resistant to prompt injection attacks. So let's dive into some of these. How much faster is it really? Well, if you take GPT-4 and its response time per token, it's about a hundred and ninety-six milliseconds, give or take, from the OpenAI API.
- 2:00
On the same API, GPT-3.5 is seventy-three milliseconds. That's three times faster.
- 2:07
How much cheaper is it? Well, taking an example with GPT-4 versus GPT-3.5 fine-tuned, you can save eighty-eight point six percent. Well, then how much shorter do the prompts actually get?
- 2:19
Well, I'll give you one example because it's gonna vary depending on your prompt. But here's what a typical engineered prompt might look like. It has some instructions, um, saying that, you know, write a blog post on this topic, how to write it, what tone to use, what to do, what not to do.
- 2:36
Well, with a fine-tune model, it learns how we write, so we don't need all of those instructions. It learns from our training examples. So we're just giving it the one thing that's unique about this prompt versus another prompt, which is the topic that we want to write on.
- 2:51
And in this very conservative example, it's ninety percent shorter.
- 2:57
Now let's talk about collaborating as a team, right? Because none of us work in a vacuum. We work with other people. Imagine a GitHub repo. You have one file.
- 3:06
Your whole code base is just one file. That's like your epic prompt. Well, with fine-tuning, now you can have multiple files like we're used to, where developers can work on this section of code or that section of code.
- 3:17
But we're not talking about code, we're talking about training examples. So your training data is this layer that your team can work on and add to and edit and improve, and then that feeds into the fine-tune model.
- 3:31
So the main point is if you can get equal or better output, why wouldn't you fine-tune a model? Now, fine-tuning is kind of a dev job right now, okay?
- 3:42
Let's be real. If you go online and you look up how to do fine-tuning, you're gonna find articles that talk about how to spin up GPU servers for training and inference, and you gotta format your data with these ad hoc Python scripts and configure these parameters and then make API calls.
- 3:57
It just looks like a dev job. But if you really break it down, why can't we just automate all of that with a user interface? Is that possible?
- 4:08
It is possible, and the bar is lower than most people think to get started doing this. If you can get twenty examples of what you want your fine-tune model to do, you can fine-tune a model.
- 4:19
This is not traditional machine learning, where you need thousands of examples to get started, and the dataset is this impossible barrier to get past. No, this is something that you could handwrite these if you want to.
- 4:32
One way to think about this is as an extension to few-shot learning. Let's say you can have five examples of what you want a model to do in your prompt.
- 4:41
Well, with fine-tuning, your training example dataset can be as long as you want. So instead of five examples, you can now have twenty or a hundred. So it seems intuitive that with more examples, the model would be able to do closer to what we want it to do.
- 4:56
So here's what I propose for a dev life cycle for large language models. We start with prompt engineering. Prompt engineering is a powerful tool. It allows us to create a prototype, to validate the concept, and we can also use it to create our initial datasets for fine-tuning.
- 5:12
Once we have those datasets, we should fine-tune a model, and we should evaluate it to make sure that it actually is better than the prompt engineered version. And then we can test which models we can get to perform at the same level.
- 5:25
Then the fine-tune model can go into production. And from production, we can capture feedback from our users, and we can log the examples. And with those examples, we can continuously improve our fine-tune model because now all of a sudden we have, um, the real examples that we can add back into our dataset.
- 5:42
So in terms of roles, I think that there's a huge opportunity for people to get into prompt engineering and fine-tuning who are not developers. Yes, if you're a developer, you can fine-tune.
- 5:52
Absolutely. But you shouldn't have to be the only person that can fine-tune. I'm a co-founder at Entry Point, and we have built the modern tooling to make this easy.
- 6:02
Let's take a look at how it works. Here we are on the dashboard, and I'm gonna open the Press Release Writer project. Let's take a look at my 20 examples.
- 6:11
Um, the way I created these 20 examples for a press release generator was I went online and I found 20 press releases that looked really good. They came from blog articles about the best press releases that you can write.
- 6:24
However, I didn't have input data, so my dataset was incomplete, but I used ChatGPT-4 to take the press release and then write a list of facts that would be needed to actually have a professional writer write such a press release.
- 6:40
You know, large language models aren't great at facts, so providing it the facts as the input makes sense to me, that I want to give it a list of facts and then have it write something that's really polished that would be a really good first draft of a press release.
- 6:52
With this user interface, I have a lot of visibility into the data that I'm actually putting into my fine-tuned model, which I think is really important. And the way this works is that we have a structured data approach.
- 7:03
So when you import, like, a CSV into Entry Point, each column becomes a field. Here I have the facts, and here I have the press release. And these fields you can use in a template.
- 7:15
Just like you were writing a mass email and you wanted to insert somebody's first name or personalize the emails with information about a contact record, you can use references to these fields with the Handlebars templating language.
- 7:28
Um, it provides a really intuitive way to easily format your output, your input. And GPT-3.5 Turbo, when you fine-tune it, you can actually use the system prompt, uh, which is where you can include instructions as well, which creates this really interesting hybrid between prompt engineering and fine-tuning, where you can have a small dataset for fine-tuning, but you
- 7:48
can also give it some instructions to help. Once we have a dataset like this, we can go and we can go to our fine-tunes, press the Add button, um, select the model, the platform, because this is cross-platform, and then we count your tokens and estimate your cost for you.
- 8:04
This is gonna be a whole dollar, so [inhales]
- 8:07
hold on tight. Press Start, and that will get started. But I have some here that are already trained, um, so let's go into one and use Entry Point Playground and see if we can actually generate a press release with our fine-tune model.
- 8:21
The list of facts here I actually wrote about, um, the AI Engineer Summit, and we'll see if we can make a press release for the AI Engineer Summit.
- 8:30
Let's go. All right. So this fine-tune model created a title here, and it made it look like a press release. What I found to be a really cool workflow is to actually create a list of facts and then generate an article, read the article, and then get ideas from it, and go back to my list of facts
- 8:49
and refine those, and then that actually becomes an iterative process to get really cool results. So I really enjoy fine-tuning. It takes a lot of the boilerplate out of the prompt, and you can just focus on what's important for the results you want, um, and the rest is taken care of by your training data.
- 9:04
Entry Point has a lot of other cool features, like data synthesis and tools to compare the performance of your fine-tune models. Unfortunately, we don't have time to go into all of that today, but I hope you will check it out.
- 9:15
It's entrypointai.com, and it was a pleasure speaking to you. [upbeat music]