# The LLM Triangle: Engineering Principles for Robust AI Applications

Almog Baku

AI Engineer Summit 2025

[Talk page](https://ai.engineer/talks/ro5HkZvzfiQ-llm-triangle-engineering-principles-robust-ai) · [Video](https://www.youtube.com/watch?v=ro5HkZvzfiQ)

## The LLM Triangle: Engineering Reliable Applications

Reliable LLM applications start with an explicit procedure, then match models, software and contextual data to each step of the work.

Basic familiarity with LLM prompts and API calls is helpful; the Python example only constructs prompt text.

### Why does the proof of concept break?

[0:41](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=41s)

What separates an impressive LLM prototype from an application that works reliably for customers? Small workflow enhancements and wrappers are an early use of a much broader computing resource. Almog Baku suggests thinking of LLMs alongside CPUs and GPUs: a new resource whose capabilities should influence how we design software. His quantum-computing analogy pushes this toward changing programming languages, but that part is too categorical—[Cirq](https://quantumai.google/cirq) already supports quantum programming in Python. The useful distinction is that LLM applications combine conventional languages such as Python, JavaScript and Go with instructions written in natural language.

![Iceberg slide with the text “Think of LLMs as a new computing resource, just like CPUs” and a circular presenter inset.](https://ai.engineer/api/talk-essay-media/375319bdba8ba5a4d81864936ef0c930a474b268f4ef9f0a96067c74b32b5615)

LLMs as a new computing resource: the tip of the iceberg.

That combination makes a proof of concept easy to build and consistent production quality difficult to achieve. The prototype works on the cases its builder has tried; then a real customer supplies the exceptional input that breaks it. The customer sees a failed application, regardless of how impressive the underlying model is. Baku describes LLM-native applications as 10% sophisticated models and 90% experimentation and data-driven engineering, immediately acknowledging that he made up the split. The practical thesis is **reliability comes from engineering the application around the model**.

Baku grounds the framework in experience founding companies, including one in AI infrastructure, and founding [GenAI Israel](https://linktr.ee/llm.org.il), which he describes as Israel’s largest such community. At the time of the talk, Baku reports 5,000 engineers, researchers and data scientists in the community. He also reports helping customers build dozens of applications over the preceding two years. The framework distills that application-building experience into a repeatable way to approach the work.

### Define the procedure before choosing the implementation

[4:28](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=268s)

The LLM Triangle connects three choices: **the model, the engineering techniques and the contextual data**. A fourth element guides all three: the **standard operating procedure**, or SOP. Borrowed from manufacturing, an SOP specifies how to carry out routine work while maintaining a required level of quality. It gives the application a process to implement rather than merely a desired outcome.

Consider a fulfillment center. A package that comes apart creates support costs, lost goods and reputational damage. The center cannot depend on every packer independently discovering the right method. Instead, it provides a sequence:

1. Check that all the items are present.
2. Mark them in the system.
3. Apply the label.
4. Inspect the package.

Baku’s analogy is to treat the LLM as a smart, agreeable intern who is inexperienced at this particular job. Intelligence does not supply knowledge of your organization’s procedure. Give the model an explicit recipe for the task.

For a landing-page builder, writing that recipe starts with an expert interview. Ask how the expert approaches a new page and which elements matter. Then describe the process back to them. Their corrections reveal where your description has skipped something they consider obvious. This decomposition reduces the cognitive load of the task while exposing the decisions the application must reproduce.

A first description might jump directly from gathering information to writing the page. The expert, however, constructs a compelling story between those two steps. That story determines what the landing page should communicate. If the SOP omits it, the implementation asks the model to make an important reasoning jump without guidance. Experts may not notice that they perform these intermediate steps, so extracting them requires specific questions. Once the steps are explicit, draw the process as a graph when loops or recursion make a simple list inadequate.

### Implement each step with the right tool

[9:03](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=543s)

Once the SOP describes what to build, engineering techniques provide ways to implement it. Baku groups them by where the work happens:

| Layer | Techniques |
|---|---|
| Prompt | Chain of thought, role assumption, few-shot examples |
| Software | Chains, tools, fine-tuning |
| Across both | [ReAct](https://arxiv.org/abs/2210.03629), structured output |

The point of this grouping is to choose a technique according to the step it must accomplish. A prompt technique alone need not carry the entire process.

**LLM-native architecture** is the arrangement of those steps into a working system. Flow engineering, chains and workflows describe much the same architectural activity here: split the SOP into components, each responsible for a particular transformation. Some components call an LLM; others are ordinary code.

A Wikipedia-style article writer makes the division concrete. Its SOP begins by identifying the subject, researching the literature, writing a table of contents and expanding each section. The implementation adds the necessary intermediate operations:

1. Normalize and distill the user’s request.
2. Categorize the subject.
3. Search Google for relevant information.
4. Convert the retrieved HTML into Markdown using code.
5. Give the processed material to an LLM component that writes the table of contents.
6. Expand the sections.

Baku motivates the HTML conversion with his observation that LLMs work better with Markdown than HTML. The architectural lesson does not require every operation to use a model: use software to prepare the information before asking the model to reason or write with it.

An LLM’s capabilities do not make it the best implementation for every step. Where a transformation can be handled deterministically, conventional code provides consistent behavior and a more direct way to inspect what happened. Reserve model calls for the parts that benefit from them.

### Choose where the agent gets discretion

[12:06](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=726s)

In this framework, an **agent** is a standalone component of the architecture that calls an LLM: a model plus a prompt template. That is a deliberately broad definition. An agent may be ephemeral, reusable or recursive; its basic responsibility is to take the process from one state to another. An **autonomous agent** adds discretion over how to accomplish the task, using tools to plan and act.

An assistant can use either design. In the autonomous version, give the model weather and retrieval-augmented generation, or RAG, tools and let it decide how to complete the request. In the crafted version, the application first asks an LLM to classify the message into an explicit `enum`: emotional support, weather, trip planning or support. That selection determines the next process, such as retrieving knowledge or fetching weather information. A subsequent LLM component composes the response; only that final response is shown to the user.

The autonomous version looks cleaner because the developer writes functions, exposes how to call them and delegates the sequence. It is quick to build and leaves room for creative solutions. The cost appears when behavior varies: an excellent solution on one run may become a failed customer demonstration on another, with a difficult debugging trail. A crafted flow makes the next step explicit, improving predictability and making failures easier to locate, but takes more development time and resources.

| Design | Main advantage | Main cost |
|---|---|---|
| Autonomous tool use | Fast to build; flexible solutions | Variable behavior; harder debugging |
| Crafted flow | Explicit sequence; easier inspection | More implementation effort |

These are trade-offs in how the application controls its work, not a reason to choose one design for every task.

Baku recommends **scoping autonomy inside clear boundaries**. Give an autonomous component an area in which it can be creative while keeping the surrounding process constrained. If that component produces a poor result, the consequences remain limited to its assigned area. The architecture can therefore combine predictable orchestration with discretion where discretion is useful.

### Match the model to the task and its constraints

[16:29](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=989s)

Different steps need different capabilities. Summarization and entity extraction generally demand less from a model than complex reasoning. Model selection must also account for the environment in which the step runs:

- **Infrastructure and performance:** Self-hosting an open model brings infrastructure work; proprietary services from Anthropic or OpenAI offer a different operational arrangement. Latency matters alongside capability.
- **Cost-effectiveness:** Compare the model’s price with the value the task creates and the budget available.
- **Data availability:** Labeled examples may support few-shot prompting or fine-tuning. A task that instead depends heavily on pretrained knowledge places different demands on the model.

The decision is therefore about a task, its data and its operating constraints—not just a model ranking.

![Factors in Model Selection slide listing four categories with examples covering reasoning, deployment, latency, pricing, budgets, fine-tuning, and in-context learning.](https://ai.engineer/api/talk-essay-media/4d209873c699196f630b968286a5075a8180968f4cb7adc5caa28bddb4f8e722)

Model selection factors: task complexity, infrastructure and performance, cost-effectiveness, and data availability.

Baku’s model-selection heuristic favors large models for broad capability, with **o1** among his examples, while recognizing that capability does not imply low cost or low latency. Smaller models are candidates for simpler tasks, especially when useful task data is available. They are more likely to struggle with complex work; a large model is a stronger starting point for a complex task with a limited dataset.

| Situation | Starting heuristic |
|---|---|
| Simple task with useful examples | Consider a smaller model |
| Complex task with little task data | Start with a larger model |
| Broad capability needed | Expect cost and latency trade-offs |

This is a selection guide, not a measured comparison of the models named in the talk.

The optimization sequence follows from that uncertainty:

1. Start with a large model and establish how the application behaves.
2. Collect data from the task.
3. Add useful few-shot examples, potentially selected dynamically.
4. Optimize incrementally, considering fine-tuning or a model change once the accumulated evidence supports it.

Baku places fine-tuning later—potentially after months of learning about the application—rather than making it a prerequisite on day zero. Early model capability buys room to understand the task before taking on that additional work.

### Build the input the model actually needs

[19:46](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1186s)

Data is the triangle’s central practical concern. Baku explains model input through a simplified picture: application software assembles instructions, user messages and context into an input that the model consumes. That picture is useful for thinking about prompt construction, but it is not a portable API contract. The current [Claude Messages API](https://platform.claude.com/docs/en/api/messages/create), for example, uses `user` and `assistant` messages with a separate top-level `system` parameter; its documentation does not establish the hidden serialization described in the talk.

A **prompt template** separates reusable instructions from the values supplied for a particular request. Substitute different variables and the application produces a different concrete input while retaining a maintainable template. The template itself does not create memory: a stateless call needs the relevant information included in its input. Organizing data into variables helps the application manage that input; it does not remove the need to decide what the model should receive.

That decision is more demanding than finding one fact in a long document. A needle-in-a-haystack test asks whether the model can recover a hidden item. Baku questions whether success on that test captures application needs: real work may require many relevant facts from a much larger collection. Too much broad context makes the task harder to focus; too little leaves the model without the information needed for a correct answer. **Supply enough context to solve the task, and select it for relevance.**

### Use examples to specify the intended categories

[22:51](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1371s)

Consider the instruction to classify a product by its name. Given `PlayStation`, the model returns `gaming console`. That is a reasonable answer, but the instruction has not specified the level or vocabulary of classification the application wants. Few-shot examples can communicate that convention through **in-context learning**, without fine-tuning.

Keep the instruction the same and add `TV → consumer good` and `printer → business equipment`. In Baku’s example, the answer for `PlayStation` changes to `consumer electronic`. Both answers describe the product, but the second is closer to the intended vocabulary. The examples supply information that the original instruction left ambiguous.

The same prompt construction can be expressed directly in Python. The operation below adds the example pairs while preserving both the instruction and the product being classified:

```python
instruction = "Classify the product by its name."
product = "PlayStation"
examples = [
    ("TV", "consumer good"),
    ("printer", "business equipment"),
]


def classification_prompt(product, examples):
    lines = [instruction]
    lines.extend(f"{name} -> {category}" for name, category in examples)
    lines.append(f"{product} ->")
    return "\n".join(lines)


prompt_without_examples = classification_prompt(product, [])
prompt_with_examples = classification_prompt(product, examples)
```

These strings are inputs for a model call; the code does not itself assign a category. The teaching mechanism is to show the desired convention through examples when describing it in words is difficult. Baku compresses that principle into “show, don’t tell.”

Adding examples without changing the request
The added pairs communicate a classification convention; the product and instruction stay unchanged.
Constructed example: Arrow formatting and object IDs are teaching representations matching the adjacent Python code. The comparison shows prompt construction, not an executed model call.
Product — unchanged: PlayStation
Operation: Insert the two demonstration pairs into the classification prompt while preserving the instruction and product.
- Instruction
  - Before: Prompt without examples: Classify the product by its name.
  - After: Prompt with examples: Classify the product by its name.
- TV example
  - Before: Prompt without examples: Not present
  - After: Prompt with examples: TV -> consumer good
- Printer example
  - Before: Prompt without examples: Not present
  - After: Prompt with examples: printer -> business equipment
- Classification request
  - Before: Prompt without examples: PlayStation ->
  - After: Prompt with examples: PlayStation ->
The added pairs communicate a classification convention; the product and instruction stay unchanged.

### Select examples for the current request

[24:39](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1479s)

A large example collection does not need to become one large prompt. **Dynamic few-shot selection** chooses examples for the current use case. Baku suggests selecting five or seven examples rather than supplying the entire collection. Similarity-based selection can use vector search, much like RAG; rule-based selection can use keywords or categories. In either case, the application retrieves demonstrations of how to perform the task, choosing the subset most useful for this request.

This returns the triangle to its organizing relationship: the model, engineering techniques and contextual data all operate under the SOP. The procedure specifies the work; engineering techniques implement and specialize it; the data gives the model what it needs at each step.

![Green triangle on concentric circles with vertices labeled Model, Engineering Techniques, and Contextual Data, a center labeled Optimal LLM Usage, and Standard Operating Procedure around the top.](https://ai.engineer/api/talk-essay-media/b3fa0795a4b45e92a373878bc65338f9b5b1299a6af8bff27bd1b4e11d378fbf)

The LLM Triangle connects model, engineering techniques, and contextual data within a standard operating procedure.

Baku closes by putting data first. Inspect the data available to the application, then use the SOP and engineering techniques to shape what reaches the model. Better task-specific input is the practical route toward better output. He ends by inviting questions through LinkedIn and pointing viewers to the full white paper; his [website](https://almogbaku.com/) lists his publications.

## Resources

- [How to Build LLM-native Apps with The Magic Triangle Blueprint](https://home.mlops.community/public/videos/how-to-build-llm-native-apps-with-the-magic-triangle-blueprint) — Baku's earlier presentation explains SOPs, prompt techniques and relevant context, with a public transcript and discussion.
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629) — The original paper on interleaving reasoning with actions that interact with external information and environments.
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://arxiv.org/abs/2404.06654) — A long-context benchmark covering multiple-needle retrieval, multi-hop tracing and aggregation beyond simple needle-in-a-haystack tests.
- [Claude Messages API reference](https://platform.claude.com/docs/en/api/messages/create) — Current request structure for Claude messages, system instructions and stateless conversations.

## Slides

### LLM-native application engineering

Against a race-car illustration, the speaker assigns 90% of LLM-native app work to experimentation and data-driven engineering. He says he made up the split.

[3:11](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=191s) [A quotation emphasizes experimentation and data-driven engineering, above an illustration of a race car.](https://assets.ai.engineer/sites/aiecode2025/sha256/d7265bbf08687afb4ad7807730c59932afeb3ae4909006b0b818242158d2cc4b.webp)

“LLM-Native apps are 10% sophisticated model, and 90% experimenting data-driven engineering work.”

4

### Who are you?

Almog Baku founded GenAI Israel and says he has helped customers build dozens of applications over the past two years.

[3:41](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=221s) [Almog Baku’s professional introduction and GenAI Israel branding.](https://assets.ai.engineer/sites/aiecode2025/sha256/446e4a305160858f0ab6f659be1b336747979d8276497dd6441de6516b84b8c3.webp)

Who are you?

Almog Baku
Seasoned tech entrepreneur; Founder @ GenAI Israel community; F-CTO for LLM; author of LLM Triangle principles; creator of RaptorML; co-founder & CTO @ Rimoto (acquired); Kubernetes maintainer.

GenAI Israel
LLM.org.il

Link[illegible]
what[illegible]

5

### Standard Operation Procedure (SOP) for LLMs

Baku uses a fulfillment center as an example: workers follow instructions to check items and label packages, helping maintain quality in routine work.

[5:04](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=304s) [A definition describes an SOP as step-by-step instructions for consistent routine work.](https://assets.ai.engineer/sites/aiecode2025/sha256/860710ca81c07847343692735495aac920b4c56a6a3ed8175c33369a64320670.webp)

Standard Operation Procedure (SOP) for LLMs

“A Standard Operating Procedure (SOP) is a set of step-by-step instructions compiled by large organizations to help their workers carry out routine operations while maintaining high-quality and similar results each time.”

8

### SOP for LLMs

Treat the LLM as a smart but inexperienced intern: give it a step-by-step recipe for the task.

[6:16](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=376s) [Slide advises treating an LLM as an intern and writing a step-by-step prompt.](https://assets.ai.engineer/sites/aiecode2025/sha256/f584e6de190f74ac16a5a3863000d11376fde1a1bab6c821cee228489c8f1a55.webp)

Standard Operation Procedure (SOP) for LLMs

• Think of the LLM as an intern
• Your prompt should be a step-by-step recipe, explaining how to excel in your task

9

### Landing Page Builder SOP

The landing page builder illustrates cognitive modeling: draw on expertise or interview an expert, then break the task into smaller steps.

[6:51](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=411s) [Slide introduces cognitive modeling for a landing page builder SOP.](https://assets.ai.engineer/sites/aiecode2025/sha256/e52843fb13c2c26cc1206e530ee2cbc616c765fce66d42fbcdff8b153c7159e2.webp)

SOP
Landing Page builder

Cognitive modeling

### Landing Page Generation Workflow

The workflow turns a language query into a landing page through concept generation, component selection and HTML composition. Failed quality checks loop back to input data reflection.

[8:45](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=525s) [Flowchart moves from a language query through page generation and HTML composition to quality checks and output, with a failure loop.](https://assets.ai.engineer/sites/aiecode2025/sha256/9671f5f53693ef0496632d10b98bc5b029361a5c33303e5dbcb0e79d0ea8617b.webp)

Native language query
Input data reflection
Generate landing page concept
Select appropriate components
If fails
Generate content for each component
Compose the final HTML
Run quality checks
Output: Landing Page

16

### Engineering Techniques

Once the workflow is defined, engineering techniques provide tools to implement the SOP and get more from the model.

[9:12](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=552s) [Engineering techniques help implement an SOP and get more from the model.](https://assets.ai.engineer/sites/aiecode2025/sha256/d2f8446709cd80dcf524f44bfedeef065d24beffe602fabaf8b10f83aafaccdf.webp)

Engineering Techniques

The Engineering Techniques help you practically implement your SOP and get the most out of the model.

18

### Engineering Techniques: Software and Prompt Layers

The diagram separates software-layer techniques, such as chains and tools, from prompt-layer techniques, such as roles and few-shots. ToT, ReAct and structured output sit in the overlap.

[9:20](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=560s) [A Venn diagram places chains, tools and fine-tuning in the software layer; chain of thought, roles and few-shot examples in the prompt layer; and tree of thoughts, ReAct and structured output between them.](https://assets.ai.engineer/sites/aiecode2025/sha256/efc510ceba1945a48ec28831072f430a781c6ff105fd094903804927a0f804c4.webp)

Engineering Techniques

Apply techniques to improve the results:

chains
ToT
CoT
tools
role
ReAct
fine-tune
Structured output
few-shots
etc.
etc.
etc.

Software layer
Prompt layer

19

### Engineering Techniques: LLM-Native Architectures

The speaker introduces LLM-native architecture as the first of two techniques, noting that others call it flow engineering, chains, or workflow.

[9:49](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=589s) [Section slide introducing LLM-native architectures.](https://assets.ai.engineer/sites/aiecode2025/sha256/c9e733c840ca9890c6142fc7ff22ca0a66c92c443ce604d64bc9179f4f26082f.webp)

Engineering Techniques

LLM-Native architectures

### Wikipedia Writer SOP and LLM-Native Architecture

The Wikipedia writer splits its SOP across LLM agents and deterministic code: agents interpret the request and write sections, while code retrieves sources and reformats them as Markdown.

[10:31](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=631s) [A Wikipedia writer SOP is mapped to a workflow that normalizes and categorizes a request, retrieves and reformats content, writes a table of contents, and expands each section.](https://assets.ai.engineer/sites/aiecode2025/sha256/2d669bbc1cd0ab9a8feb89b432813cd40133e8eb0b4f60a0ddc82f6eaa1daa6b.webp)

Wikipedia writer SOP
1. Start by understand the real subject
2. Research for literature
3. Write the table-of-content or the subtitles of our sections to ensure a fluent narrative
4. Expand on each section

LLM-Native Architecture
Normalize and distill user request
Categorize the subject
Retrieve relevant written content from the internet
Reformat to Markdown
Search engine
Write the table of content
Expand on each section

deterministic code
LLM Agent

22

### Agent

An agent combines an LLM with a prompt template. Agents can be ephemeral, reusable, or recursive; adding information to the same instance keeps it the same agent.

[12:08](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=728s) [An agent is presented as a standalone component that calls an LLM, with an LLM and prompt template forming an agent. Agents can be ephemeral, reusable or recursive.](https://assets.ai.engineer/sites/aiecode2025/sha256/80db775885b2fa85220695bbb9120cbb0c60eac11d01cd40a912387bde0ec51b.webp)

Agent

“An LLM agent is a standalone component of an LLM-Native architecture that involves calling an LLM.”
~ The LLM Triangle Principles

Instance of: LLM + Prompt (tmpl) = Agent

Not all agents are equal: some are ephemeral, some resable, some recursive...

If we add information to the same instance, it’s the same agent :)

### Agent

An agent takes a task from A to B by doing something. An autonomous agent can decide how to carry out that task.

[12:40](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=760s) [Agent: “Take me from A to B by doing XYZ.”](https://assets.ai.engineer/sites/aiecode2025/sha256/3fdeca9e0062e3f792a3fb51093783fc12ed98c379d6b75b5564f1646dfc06f6.webp)

Agent

“Take me from A → B by doing XYZ.”

### Match the Right Model to the Mission

The speaker turns from engineering techniques to model selection: choose a model for the task, considering whether it calls for summarization or complex reasoning.

[16:38](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=998s) [A slide advises matching the model to the mission because LLMs differ.](https://assets.ai.engineer/sites/aiecode2025/sha256/6d967226fc407c9f0e99251241467ae325dda38fa4552570d276788db792b1e2.webp)

Match the right model to the mission

“Not all LLMs are created equal. Match the model to the mission.”

31

### Factors in Model Selection

Compare a model’s price with its capabilities, and weigh cloud versus edge deployment against latency. Labeled data may support fine-tuning; examples can also be supplied in context.

[16:43](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1003s) [Factors include task complexity, infrastructure and performance, cost, and data availability.](https://assets.ai.engineer/sites/aiecode2025/sha256/432b5c1269e0268f5328b81ea8fb4b1b1e625597192beeb426427b9ca62a05e0.webp)

Factors in Model Selection

1. Task Complexity
   a. Simple tasks (e.g., summarization) vs. Complex reasoning
2. Infrastructure & Performance
   a. Cloud vs. Edge deployment
   b. Latency considerations
3. Cost-Effectiveness
   a. Pricing vs. Capabilities
   b. Predicted usage and budget constraints
4. Data Availability
   a. Labeled data for potential fine-tuning
   b. In-context learning capabilities

32

### Model Size and Task Requirements

Small models can suit simpler tasks or data-rich settings, but may underperform on complex tasks. Large models can handle complex tasks with limited data, though they may be slower and more expensive.

[18:06](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1086s) [A four-quadrant chart compares small and large models for simpler and complex tasks and different data availability.](https://assets.ai.engineer/sites/aiecode2025/sha256/3b051869fa8162f2eef62b286cd555e03b1c102b9a530c016fb6c55e63323404.webp)

Task complexity / Dataset availability
Model size

Small models
simpler tasks / rich dataset

Large models
Viable but expensive

Small models
Underperform

Large models
complex tasks / limited dataset

33

### How to Start?

Start with a strong model and collect data to understand the application. Use few-shot examples, experiment with smaller models, and consider fine-tuning last.

[18:59](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1139s) [Start with a strong model, collect data, use few-shot learning, test smaller models, and consider fine-tuning last.](https://assets.ai.engineer/sites/aiecode2025/sha256/49feaefc3a80d70a2b92f4e811117fd3891f022067664dd75bc0c83f62a8bebf.webp)

How to start?

“Start big, collect data, optimize incrementally”

• Start with a Strong Model
• Collect Data
• Leverage Few-Shot Learning
• Experiment with Smaller Models
• Consider Fine-Tuning as a Last Resort

34

### Relevant Data: The Key to Specificity

Give the model enough task-specific context while filtering irrelevant data. The speaker warns that too much information can hide useful details, while too little can prevent a correct answer.

[21:30](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1290s) [A balance diagram illustrates the tradeoff between too little information and too much context.](https://assets.ai.engineer/sites/aiecode2025/sha256/1b9db404f6e981a0a476c096558858eb155bb151e530ef70d94894c995bab6e9.webp)

Relevant Data: The Key to Specificity

1. Minimize irrelevant context to avoid the “needle in haystack” problem
2. Embed the relevant context information for the specific task

Lack of Info
TMI

40

### Few-Shot Learning

Examples steer the product classifier toward the desired category: PlayStation changes from “gaming console” in the zero-shot example to “consumer electronics” with examples in the prompt.

[22:49](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1369s) [Side-by-side product-classification examples compare a zero-shot prompt with a prompt containing examples.](https://assets.ai.engineer/sites/aiecode2025/sha256/df569d5393b940e5519a5add876c38992820e543906a6a50eae733f2e1b8b896.webp)

Few-shot learning

Zero-Shot
System: classify the product type by its name. only respond with the type
User: playstation
AI: gaming console

Few-Shot
System: classify the product type by its name. only respond with the type
User: tv
AI: consumer good
User: printer
AI: business equipment
User: playstation
AI: consumer electronics

43

### Dynamic few-shot

An example selector chooses a handful of examples for each use case instead of sending the whole collection. Selection can use similarity search or rules such as keywords and categories.

[24:41](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1481s) [An example selector points toward a list of examples.](https://assets.ai.engineer/sites/aiecode2025/sha256/fdb89b4d179565d6dda4617685d46eb64dd9c40f89ba6198d345094a678f9aaa.webp)

Dynamic few-shot

Example Selector

Example
Example
Example
Example
Example
Exam[illegible]
Exa[illegible]
[illegible]
Exam[illegible]
Example[illegible]

44

### The LLM Triangle Principles

The triangle connects the model, engineering techniques, and contextual data, with a standard operating procedure guiding all three. The speaker recommends adapting data with engineering techniques before feeding it to the model.

[25:24](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1524s) [A triangle connects model, engineering techniques, and contextual data, surrounded by a standard operating procedure.](https://assets.ai.engineer/sites/aiecode2025/sha256/ed8e5b4cef7746b88e8f8d6c4b6d30d582d7c799c9fa60a74f487cff2954d334.webp)

The LLM Triangle Principles

Standard Operating Procedure
Model
Engineering Techniques
Contextual Data
Optimal LLM Usage

47

## Full transcript

[0:00](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=0s) Hey, thank you for having me, and thank you for being here today. Here, we're going to talk about the LLM Triangle Principles, engineering principles for building robust AI applications.

[0:14](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=14s) My name is Almog, and I'm gonna talk with you about, surprise, surprise, LLMs. So this is a prerecorded session, but I think it's a safe bet to say that everyone, everyone told you today or gonna to-- tell you today that LLMs are amazing.

[0:33](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=33s) They're gonna change the world. They, they basically gonna shift the way how we think about technology and software.

[0:41](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=41s) And to be honest, most of the use cases today are just like sparkers, like small projects that a little bit enhance the workflow.

[0:55](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=55s) And regarding to what they said, I totally agree with them. LLMs are amazing. They're blowing my mind.

[1:02](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=62s) And today, this is just the beginning. This, this is just the tip of the iceberg.

[1:09](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=69s) Most of the use cases today might be LLM wrappers or GPT wrappers, but the future

[1:17](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=77s) gonna blow your mind. But in order to get there, we need to shift our thinking. Uh, I think that a good analogy for that is thinking of LLMs like a new resource, like GPU or CPU, and Google just showed us today that quantum computer is around the corner.

[1:36](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=96s) Um, actually, it wasn't today. It was a few weeks ago, but never mind. Um,

[1:41](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=101s) but everyone know that when quantum computer will be here, nobody will write Python to program quantum computing applications. So why do we do that with LLMs? Um, I think this is very confusing, uh, because we do build LLM applications with Python or, I don't know, resembling JavaScript or

[2:06](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=126s) PHP. Just kidding. Amazing languages. Not really. Yes, they do. Um,

[2:14](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=134s) y- we do write them with Python and JavaScript and Go and whatever, but we also write them in natural language. And I guess that everyone here that tried to build something or built something know that building POCs is very easy.

[2:35](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=155s) But production applications that's performing in a certain quality, that's damn hard. Because every time we take the POC and show that in our demo to a real customer, every time, we just facing the only use case that makes everything break, and the customer is just d- don't get it.

[2:58](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=178s) Like, "Why the hell are you wasting my time?" So

[3:03](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=183s) over the past few month, um, past few re- years, I've been thinking about it, and I, I came to the realization that LLM-native applications are ten percent sophisticated models and ninety percent of hard experimentation and data-driven engineering work.

[3:22](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=202s) And trust me about that, I'm an engineer. I just made this up. Uh, uh, so, uh, let's wrap it up. So my name is Almog. I'm a serial entrepreneur.

[3:33](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=213s) I started a few companies. My previous company was in the AI infrastructure world. I also the founder of GenAI Israel community. Uh, we are the largest community in Israel.

[3:45](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=225s) We have five thousands engineers and researchers and data scientists working on GenAI applications. And over the past two years, I've been helping to build dozens of applications, uh, for my customers.

[4:00](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=240s) Um, and over the next few minutes, I'm gonna help you, uh,

[4:08](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=248s) to maybe, uh, uh, see the way I see about applications and learn from my experience. So I'm trying to distill my experience into, I don't know, the next fifteen, twenty-ish minutes, and to help you to reproduce what I did.

[4:28](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=268s) All right. So the LLM Triangle Principles build upon a few principles. Uh, we have the model, obviously, the foundation model, the engineering techniques, and the data. And all of that are guided by, of course, the fourth principle of the triangle, which is the standard operating procedure.

[4:55](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=295s) If you haven't heard about standard operating procedure, that's totally fine. That probably meant that you haven't worked in a huge corporation or a factory. So to be fair, I borrowed this terminology from the info-- uh, uh, manufacturing industry, and it basically means that standard operating procedure is a set of step-by-step instructions compiled by

[5:19](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=319s) loud, by large organizations to help you, uh, to help the worker to carry out a routine work while maintaining a certain quality. So let's think about a huge fulfillment center, right?

[5:34](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=334s) They can't allow themselves to hire some PhD in physics in order to wrap some package, so it won't, I don't know, get lost or, or, uh, get unwrapped, right?

[5:47](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=347s) Because if a package gets unwrapped, that cost them a lot of money, like customer support, the, the price of goods, their name, etc. So instead of that, they just hire a lot of interns and tell them exactly what you do-- what to do.

[6:04](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=364s) First, you need to check all the items are there, and you need to mark them in our system. And then you put the label, and then you make sure the, the package is all right or whatever.

[6:17](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=377s) That's exactly how we should look of the LLM, right? We should look of the LLM like an intern. He's really smart.

[6:28](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=388s) He's really, he's really nice guy, right? He, he says, "Yeah, you're right," about whatever we say.

[6:35](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=395s) And on the other hand, he's inexperienced on our ta-- at our task. So we should prompt it and tell it exactly what to do, if you will, like a step-by-step recipe.

[6:48](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=408s) This way, we can ensure it's excelling our task. So how do we do that? Uh, in order to, to think about that, let's take an example. So our example gonna be how to build a landing page builder.

[7:03](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=423s) So first, we want to assume that w-we are the expert, or maybe to interview a real expert, and to reduce the cognitive load of the task by simplifying it into some smaller steps.

[7:18](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=438s) So we'll start to interview the expert, or assume to be the expert. How would you f- approach this task? How would you create a new landing page? What's the key elements you consider?

[7:30](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=450s) And then we will reflect on it, and we'll phrase it for him. Usually, the expert will say, "Oh, no, no, that's not what I'm doing. I'm doing this and this and this."

[7:40](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=460s) And this will allow us to have more information about the nuances that the expert know, but n-not always tell us, right? So eventually we'll have, like, a list of steps, and our goal is to find the hidden implicit, uh, cognition jumps that the expert know, but we don't.

[8:02](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=482s) Because it's probably obvious that after you gather data, you write the page, but in between

[8:11](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=491s) the expert writing a compelling story that the landing page should deliver to us. And that might be obvious to him, but not for us. And this expert, she might, she might know all of the nuances because obviously everybody know them, but we don't, and we need to identify them.

[8:35](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=515s) And sometimes they will do that, and they won't even notice that they do that. So we need to be very specific and ask very good questions, or at least assume to be them.

[8:47](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=527s) Uh, all right. After that, we'll compile a list, and I found that sometimes it's very helpful to describe this process as a graph because some of the processes are like, like loop or like we have a recursion there.

[9:03](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=543s) So we-- It's very helpful to draw it. All right. So now we know what to build. Now that's, that's the time to actually build it. So the engineering techniques, that's exactly what they do.

[9:16](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=556s) It's a set of tools we can use in order to implement our SOP. So there are basically two types of, uh, engineering techniques. We have engineering techniques solely in the prompt layer, like chain of thought, role assumption, few shots, et cetera.

[9:33](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=573s) And we have some techniques that are solely in the software, right? Like chains, tools, fine-tuning. And obviously, we have everything that in between, like ReAct, structured output. And we need to try and think about what do we want to achieve and how do we achieve it.

[9:51](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=591s) I want to focus today on two critical techniques. The first one is the LLM-native architecture. All right. To simply put, it's just an architecture. All right? Some people call it flow engineering or chains or workflow.

[10:10](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=610s) It's all the same. It's simply an architecture.

[10:15](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=615s) What we do here is we take the SOP, and we split based on the SOP, the process into multiple steps, multiple agents, if you will. Every agent will do something different, and sometimes we'll have some piece of code that will do that.

[10:33](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=633s) So for example, let's say we are building a Wikipedia writer. So the SOP said that we start to understand what the real subject we want to write about, and then we do some literature with researching it, and then we write a table of content, and then we expand on each section, right?

[10:50](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=650s) So in order to build that, we'll split it into some process. So first, we normalize and distill the user request. We categorize the subject. We do some Google search and find some data.

[11:03](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=663s) And if you haven't noticed, LLMs are really good with Markdown and they are so-so with HTML. So before we're gonna deliver this information to the LLM, we're gonna, uh, uh, we're gonna process it with some piece of code that format it to a Markdown and then to another agent that write the table of contents and et cetera.

[11:25](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=685s) So you can notice that some parts are done by LLM and some parts are we process it using a regular software. So LLM is an amazing piece of technology.

[11:40](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=700s) Seriously, it's mind-blowing. That being said, it doesn't mean that we need to do everything with LLMs, right? Always [laughs]

[11:49](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=709s) LLMs, like, like a, a deterministic code will always be deterministic. It will always provide us with consistent quality. So you might notice that I said agents, agents, agents. What are agents?

[12:06](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=726s) So based on the LLM Triangle Principles, agent is simply an LLM. Uh, LLM agent is a standalone component of the LLM architecture that involve calling an LLM Simply put it, it's LLM model plus prompt equal agent, and by prompt I mean prompt template.

[12:28](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=748s) And not all LLM agents are equal. Some of them are ephemeral, some of them are reusable, some of them are recursive, and basically, uh, an agent is just take me from, from A to B by doing something.

[12:48](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=768s) And we have a special kind of el-- of agent, which is the autonomous agent. Autonomous agent are given with the liberty, with the right to decide how to implement this task.

[13:05](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=785s) So it's basically agent with some tools. So the agent can decide how to plan and how to solve the problem. So take me from A to B by doing something.

[13:20](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=800s) I don't care how, just do it. And we can implement the same problem with both ways, right? So for example, let's say we just want to build some assistant, right?

[13:34](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=814s) So we can say, "Hey, you are a smart assistant with access to the weather and RAG tool," uh, the, the, the knowledge tool, right? And the agent will just do whatever it wants in order to complete the task.

[13:49](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=829s) Or we can achieve exactly the same with an architecture, right? First, we'll process the, the, the re-- the, the message. What does the user want from us? Emotional support, to know the weather, to plan some trip, to s- to, to get some support, and we process that with some agent, uh, and then we get like, like simply

[14:12](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=852s) to... We feed the agent with enum and tell the agent, "Hey, just select one of them," and then based on each selection, we'll do different processes. One of them will fetch information from RAG, the other one maybe will fetch some weather information, and then the result of that will deliver to a thir- a, a second agent, which

[14:33](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=873s) composing the response, and only that the user gonna see.

[14:38](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=878s) And that's pretty complex. Why would we want that? Then like the, the first option is much simpler, it's much cleaner. Like life is full of trade-offs, right? There's no such magic.

[14:52](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=892s) So autonomous agents are elegant software, seriously amazing, very creative, very fast to build, to build. Like you simply write some functions and provide the LLM with the, the way to call them.

[15:10](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=910s) That being said, they are very hard to debug, and their quality might be unsustainable. Like one time you might have a very creative, amazing solution, and the other time, exactly at the point when you demoing to the customer, it's gonna fail.

[15:26](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=926s) In production, we need to have some sort of certainty, and with a crafted flow, uh, we have a sustainable quality. Uh, the process is predictable. We know what's going after each step.

[15:42](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=942s) Higher possibility to debug it, and from the other side, it's very expensive because we need to invest a lot of time and a lot of resources to handcraft and develop it.

[15:57](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=957s) My two cents is that reality, like always, depends. The way I see it is that I highly recommend to scope, uh, the autonomous agents. So, so like they can be amazing as long as they have clear borders.

[16:16](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=976s) Like this is the way-- This is the area that you can be creative. Um, so worst case, it won't deliver you good results, but it's very hedged.

[16:29](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=989s) All right, so we've discussed the engineering techniques. Now let's talk about the model. Not all LLMs created equal. We need to match the task

[16:43](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1003s) to the model. There are various factors to select the model. The complexity of the task. Uh, for example, simpler tasks like summarization or entity exa- extractions are easier for LLMs to fulfill rather than having a complex reasoning about some,

[17:05](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1025s) some thought they need to compile. Um, infrastructure and performance. Uh, a lot of models are out there open sourced and require us to build some sort of infrastructure in order to run them versus some proprietary software, which is more expensive, and we can use like the large models from

[17:29](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1049s) Anthropic or OpenAI. Also, we have the consideration of latency, right? Uh, the cost effectiveness. Uh, what's the price of the model versus how much it worth for us?

[17:46](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1066s) Um, wha- what's our budget constraints? Uh, the data available for us. If we have a lot of labeled data, maybe we can do some fine-tuning, or maybe we can feed it with some few shots.

[18:00](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1080s) But if we rely heavily on the model knowledge, then we need probably a different model. Uh, I found that this chart very helps me. Um, so all right, let's go through it very quick.

[18:15](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1095s) Large model are expensive, and they're probably good at most use cases, right? We can throw most problems on o1 or o3 or o9, and it will be amazing.

[18:29](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1109s) It doesn't mean that it will be fast and cheap.

[18:33](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1113s) Smaller models are pretty fine with simpler task or-- and when we have a lot of data to feed them.

[18:43](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1123s) Um, smaller model, uh, with a very complex task probably will underperform, and large models with complex task and limited data set will probably perform fine.

[19:01](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1141s) So what should we do? My two cent is

[19:06](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1146s) you gonna build an LLM-native application. It's not a short rodeo. It's gonna take you time. Start with the big model, collect data, understand what's going on, and then optimize it incrementally.

[19:18](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1158s) In-incrementally, sorry. Uh, that means that you start with a big model, you understand what's going on, maybe you leverage some few-shot you creating, maybe some dynamic few-shot, we'll talk about it later.

[19:33](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1173s) And maybe after a few month, you'll fine-tune or change the model. We don't need to, to, to jump into the, the heavy lifting of fine-tuning from day zero, right?

[19:46](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1186s) All right. We can't talk about LLMs without talking about data. It's all about the data. Seriously. You can forget everything and just focus on the data because this is the mot-- most important part of my talk.

[20:03](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1203s) So it's important to realize that the model doesn't really understand what context or data is, or user prompt, or even system prompt. It only get a s-- It only gets a single text, a prompt.

[20:20](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1220s) So when we use, um, OpenAI or Claude APIs, we provide it with a system prompt and a user prompt and another system prompt. Behind the scene, uh, there is a piece of software that takes all of these pieces, glue them together with some template and provide it to the model.

[20:40](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1240s) The model sees some input text and return you with some output text. That's the way it work.

[20:47](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1247s) And we [chuckles] as builders, we need to make sense out of it. So we split it. And a very important piece of concept is the prompt template. Basically,

[21:02](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1262s) um, we have some variables. We can place some variables in our template and to feed it to the model. So we have like a prompt template, but each time it's slightly different.

[21:16](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1276s) Again, the model doesn't have any memory. It doesn't understand what the context and the variables are. It's stateless. It's just a trick for us, but it help us to maintain some piece of sustainable code, right?

[21:32](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1292s) And the key is to make the data relevant, right? Maybe you've heard about the needle in the haystack problem. So like the needle in the haystack problem says that if we'll feed the model with tons of data, and we'll place some needle inside of the haystack,

[21:52](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1312s) maybe the model won't find it. And recent, uh, publications, uh, by the LLM vendors said, "Oh, now we solved the problem. Now we can find this piece of hidden needle in our text."

[22:07](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1327s) To be honest, I don't really believe them because in reality, we don't really have a needle in a haystack. We have a lot of hindle-- needles in a huge haystack.

[22:19](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1339s) So if we'll feed the model with a lot of data, and we'll ask the model to understand something, that would be very hard. From the other side, if we won't provide the model with enough information, obviously it won't be able to provide us with the right answer.

[22:36](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1356s) So we need to find the right balance, uh, of giving the model the context, but not too much a broader context, and how to focus the data we provide it.

[22:51](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1371s) Um, there are a few practices and techniques how to teach the model what to do. Um, for example, few-shots. So we can ask the model, "Hey, uh, please classify the product by its names."

[23:08](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1388s) And we say PlayStation, and the model will return us gaming console. That's right. But we meant something slightly different. By providing the model with some examples, we can basically utilize one of the greatest, uh, uh, qualities of these foundation models, their way to be in-context learners, to

[23:33](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1413s) learn something from the context we provide without fine-tuning it. So classify the product by its name, the same system prompt. But when I say TV, you say consumer good.

[23:45](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1425s) TV, consumer good, and we just play with it, right? When I say printer, you say business equipment. Printer, business equipment. When I say PlayStation, you say... And then the model suddenly use a very different language.

[23:59](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1439s) It said consumer electronic. All right, so both of the answers are correct, but the second answer is more in the sense. It's more in the vibe we are looking for, which is consumer electronic, right?

[24:14](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1454s) So that's a very good way to teach the model some concept that is very, very hard to, to describe in words, right? So we teach it by example.

[24:27](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1467s) I think that a, a, a key phrase to remember is show, don't tell.

[24:34](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1474s) Sometimes it's easier to teach by showing and not by explaining.

[24:39](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1479s) That's one of the key moments to use few-shots. Some companies took this concept even further. They said, "All right, we have tons of examples, and based off the use case, we're gonna find you the best example for w- for the model."

[24:55](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1495s) And based of the-- based on the use case, we're only selecting a handful set of examples. Not all the examples, only five or seven. Um, we can use some similarity mechanisms like vector search or something like that, much like RAG, or we can do some logic selection of the, of the, of the, of the example,

[25:21](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1521s) like by keyword or by category or whatever. Um, so this is the LLM Triangle. It, uh, composed, uh, by three, uh, key, uh, uh, principles, the model, the engineering techniques, and the contextual data, all guided by the SOP.

[25:44](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1544s) Again, data is the king. Without data, we don't have a kingdom.

[25:51](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1551s) Look at your data, try to think how you specialize it using the engineering techniques, the SOP,

[25:59](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1559s) to feed it to the model and to yield higher quality.

[26:04](https://www.youtube.com/watch?v=ro5HkZvzfiQ&t=1564s) Thank you. Uh, if you have any question, please reach out to me over LinkedIn. Um, you can read the full white paper using this link, and thank you. Have a good one.
