# Domain adaptation and fine-tuning for domain-specific LLMs

Abi Aryan

AI Engineer Summit 2023

[Talk page](https://ai.engineer/talks/cXPYtkosXG4-domain-adaptation-fine-tuning-domain-specific-llms) · [Video](https://www.youtube.com/watch?v=cXPYtkosXG4)

## Domain adaptation and fine-tuning for domain-specific LLMs

Adapting a language model requires more than choosing a fine-tuning method: the domain, training data, memory budget, evaluation, and operating pipeline all shape the result.

Familiarity with neural-network weights, attention, and the distinction between training and inference will help with the adaptation mechanisms.

### When a general model does not know your domain

[0:32](https://www.youtube.com/watch?v=cXPYtkosXG4&t=32s)

How do you make a general-purpose language model useful in a domain that its training data barely represents? Some specialist knowledge is scarce; some cannot enter public training corpora because of compliance restrictions. Abi Aryan begins with this practical gap: a broadly capable model does not automatically cover every enterprise or hobby use case. Fine-tuning is one response, but knowledge bases, retrieval augmented generation (**RAG**), and prompting also belong in the decision.

A second motivation is avoiding a fresh data-collection effort for every domain. Aryan uses language transfer to illustrate the attraction of reusable representations: embeddings can capture structure that helps a model transfer between related settings. She tentatively offers English and Latin as an example of similarity. That analogy motivates transfer, but shared structure alone does not establish that a model can reliably handle an entirely unseen language without examples. The broader goal is access: adaptation can also personalize a model for users whose needs the general model does not adequately serve.

![Slide titled “Domain Adaptation: Why do we care?” with three bullets on underrepresented domains, reducing data collection and labeling, and making models accessible to more users.](https://ai.engineer/api/talk-essay-media/c599914504c3f23bc3890cb3a6e1ae355f3e1e936e812009b91ac504cd69ebcc)

Why domain adaptation matters: specialized domains, limited data, and broader access.

### Locate the parameters before choosing an update

[3:32](https://www.youtube.com/watch?v=cXPYtkosXG4&t=212s)

**Fine-tuning changes learned parameters to improve a pretrained model for a new requirement.** The initial neural-network picture is familiar: inputs pass through hidden layers, whose weights and biases determine the outputs. Aryan then moves to an encoder–decoder transformer diagram, with attention and feed-forward networks on both sides. This is the architectural reference for the methods that follow, rather than a claim that every language model has both an encoder and a decoder. The useful question is where adaptation enters that computation and which parameters remain fixed.

### Full updates versus selective adaptation

[4:51](https://www.youtube.com/watch?v=cXPYtkosXG4&t=291s)

The first choice is whether to update all model weights or a much smaller set. Aryan connects full updates to earlier transfer-learning and teacher–student distillation work from the 2016–2018 period: a student learns from a teacher, with training changing the student's parameters. Distillation describes the source of supervision, however, rather than requiring a particular parameter-update scope. The practical pressure is compute and storage. As models grow, repeatedly training and storing complete task-specific copies becomes expensive. Selective adaptation aims to retain useful performance with less trainable state; matching **ChatGPT** is not a guaranteed consequence.

The talk organizes the next methods as adapter tuning, prefix tuning, and parameter-efficient tuning. More precisely, **parameter-efficient fine-tuning (PEFT)** is the umbrella that includes adapters and learned prefixes as well as low-rank updates. Aryan also mentions instruction tuning through examples and sets aside reinforcement learning from human feedback because collecting human judgments is expensive. Examples used for instruction tuning become training data; examples placed only in a prompt do not update weights. That distinction separates training from in-context learning.

### Add small adapter modules

[7:17](https://www.youtube.com/watch?v=cXPYtkosXG4&t=437s)

In adapter tuning, small trainable components are inserted into the existing network. Aryan's illustration adds two components containing extra weights while leaving the original transformer intact. These modules provide a place to learn domain-specific changes without updating the entire pretrained model. This is the central mechanism of [adapter-based parameter-efficient transfer learning](https://arxiv.org/abs/1902.00751).

Aryan reports matching model performance with only 0.15% of the parameters, but does not specify the task, benchmark, or parameter denominator. Treat that number as an attributed result, not a capacity-planning target. Her application example is biochemical engineering: a substantially different domain whose terminology and patterns may require adaptation. She uses “adapter” and “adaptive” fine-tuning interchangeably here; in implementation, keep the distinction between an adapter module, which is a mechanism, and domain adaptation, which is a goal.

### Steer attention with a learned prefix

[8:36](https://www.youtube.com/watch?v=cXPYtkosXG4&t=516s)

[Prefix tuning](https://arxiv.org/abs/2101.00190) keeps the base model frozen and learns continuous prefix vectors that subsequent tokens can attend to. Aryan describes this as adding an embedding layer in front of attention. Unlike an ordinary text instruction, the prefix consists of trainable values: optimization changes those values while preserving the pretrained parameters.

![Slide titled “How does Fine-tuning work under the hood?” showing model adaptation diagrams and a line chart comparing prefix-tuning, adapter-tuning, and PLM fine-tuning.](https://ai.engineer/api/talk-essay-media/3230bb407032f0b765d44bb86a57257ba4fd33418ea93eedff9415e08fbc162a)

Fine-tuning and prefix-tuning diagrams alongside a training-set-size comparison.

The analogy is a water tank and a tap. The tank supplies the water; the tap shapes how it comes out. Similarly, a learned prefix steers how the model uses its existing capabilities. Aryan also calls this a masking layer, but the literal mechanism is conditioning attention on learned vectors, not masking or replacing the base weights. The model's output behavior changes even though its pretrained parameters do not.

### Learn a low-rank update, not a smaller base model

[9:46](https://www.youtube.com/watch?v=cXPYtkosXG4&t=586s)

[**LoRA**, or Low-Rank Adaptation](https://arxiv.org/abs/2106.09685), and [**QLoRA**](https://arxiv.org/abs/2305.14314) enter the discussion through resource constraints. Aryan points to laptops and even Arduino-class devices as motivations for efficiency. Those examples express the desire to work with limited resources; reducing fine-tuning memory does not by itself make a large language model deployable on a microcontroller. Training memory, adapter storage, and inference requirements are separate budgets.

To explain rank, Aryan asks the reader to imagine matrix rows or columns that repeat information. If one is a scalar multiple of another, it does not add an independent direction. That intuition helps explain why a change to a large matrix might be represented with fewer degrees of freedom. **LoRA applies that low-rank constraint to the learned update.** It does not inspect the pretrained matrix and delete redundant rows or columns. The base weight matrix stays frozen, while two smaller matrices are trained.

For a base matrix W₀ with output dimension d and input dimension k, the forward computation is:

```math
\begin{aligned}
W &= W_0 + \frac{\alpha}{r}BA, \\
y &= W_0x + \frac{\alpha}{r}B(Ax), \\
A &\in \mathbb{R}^{r\times k},\qquad
B \in \mathbb{R}^{d\times r}.
\end{aligned}
```

Here, r is the chosen rank and α controls the update's scale. The following PyTorch module makes the frozen base and trainable update explicit:

```python
import torch
from torch import nn

class LoRALinear(nn.Module):
    def __init__(self, base: nn.Linear, rank: int, alpha: float):
        super().__init__()
        if rank <= 0:
            raise ValueError("rank must be positive")
        self.base = base.requires_grad_(False)
        self.scale = alpha / rank
        self.A = nn.Parameter(
            base.weight.new_empty(rank, base.in_features)
        )
        self.B = nn.Parameter(
            base.weight.new_zeros(base.out_features, rank)
        )
        nn.init.normal_(self.A, std=0.02)

    def forward(self, x: torch.Tensor) -> torch.Tensor:
        update = (x @ self.A.T) @ self.B.T
        return self.base(x) + self.scale * update
```

Initializing `B` to zero makes the initial update zero, so the layer initially preserves the base output. Training changes `A` and `B`, not `base.weight`. This reduces trainable parameters and their associated gradient and optimizer state; it does not automatically shrink the frozen base model. That is the precise memory benefit behind the talk's broader model-size discussion.

### Quantize the frozen weights with QLoRA

[12:11](https://www.youtube.com/watch?v=cXPYtkosXG4&t=731s)

QLoRA adds another memory-saving choice: the frozen base weights use four-bit storage, and gradients pass through that base into trainable LoRA adapters. Aryan describes a workflow of starting with a pretrained model, collecting labeled target data, and training adaptation matrices. The low-rank matrices multiply each other to form an update whose contribution is added to the base computation; they are not simply multiplied into the main weight matrix. Likewise, reducing a distance between source-domain and target-domain predictions is her description of the adaptation goal, not the defining QLoRA objective. The training loss still depends on the task and dataset.

The implementation tradeoff is support. Aryan flags dependencies such as [bitsandbytes](https://github.com/bitsandbytes-foundation/bitsandbytes), warns that device availability and cross-model efficiency testing are limited, and favors optimizing LoRA at the time of this October 2023 talk. That is historical implementation advice: current library support covers additional hardware configurations, so today's choice requires checking the actual backend and device rather than carrying forward a blanket preference.

### Prompting, retrieval, and training can work together

[13:43](https://www.youtube.com/watch?v=cXPYtkosXG4&t=823s)

The discussion now returns to the larger choice among prompting, RAG, and fine-tuning. Prompting can provide no demonstrations, one demonstration, or several: zero-shot, one-shot, and few-shot use. Aryan suggests about ten few-shot examples, citing ChatGPT rather than a task-specific experiment. She presents additional examples as helpful and prompting as cheaper and less data-intensive, while characterizing it as more suited to general demonstrations than demanding applications. Neither a fixed example count nor a blanket performance ranking against fine-tuning follows from that discussion; the examples and task determine whether the extra context helps.

**These are composable choices.** Fine-tuning can shape a model's behavior, prompting can specify the current request, and retrieval can supply relevant information at inference time. Aryan explicitly allows prompt engineering and RAG alongside fine-tuning, as well as combining domain-focused and behavior-focused training. Choosing one does not require abandoning the others.

### Separate the domain, the task, and the update budget

[14:56](https://www.youtube.com/watch?v=cXPYtkosXG4&t=896s)

Consider Aryan's legal-company example. A company may want a model that handles five or ten tasks across legal work, or it may want to optimize one specific task. She calls the first goal adaptive fine-tuning and the second behavioral fine-tuning. For the latter, the focus narrows to the task's label space and prior probability distribution: what outputs are possible, and how frequently they occur. She suggests inference and reasoning as potential uses and offers LangChain functions as an analogy for task-focused behavior, rather than as a training mechanism.

| Scope | Question it answers | Example or mechanism |
| --- | --- | --- |
| Domain adaptation | What subject area should improve? | Several tasks within legal work |
| Behavioral tuning | What specific behavior should improve? | One task and its output space |
| Parameter efficiency | How much trainable state can we afford? | Freeze the base; train a small update |

The first two rows describe the goal; the third describes how to pursue it. LoRA or QLoRA can therefore support either domain-focused or task-focused training. They are not competing objectives on the same axis.

![Comparison slide with columns for Prompting, Retrieval Augmented Generation (RAG), and Fine-Tuning; the fine-tuning column describes adaptive, behavioral, and parameter-efficient approaches. A speaker inset appears below left.](https://ai.engineer/api/talk-essay-media/a32e38ff2ca9ad526f5fdac9cbf911cf9d149f9ae0a7b03faf036b76f4411921)

Prompting, retrieval augmented generation, and fine-tuning compared in three columns.

### The adaptation method depends on the data

[16:17](https://www.youtube.com/watch?v=cXPYtkosXG4&t=977s)

Once the method is chosen, data quality becomes the limiting factor. Aryan's preparation sequence covers collection, tokenization, cleaning, normalization, and noise removal. These choices determine what information the model actually receives—not merely whether the training job can load the dataset.

Deduplication is especially consequential. Repeated records expose the model to the same material again and again, increasing its influence on training and encouraging memorized outputs. Aryan attributes much memorization to duplication; the narrower supported conclusion is that duplication contributes to it, not that removing duplicates eliminates memorization. After deduplication, she adds data augmentation to the preparation toolkit. Augmentation should expand useful variation rather than recreate the repeated exposure that cleaning was meant to reduce.

### Specialization does not guarantee broad generalization

[17:21](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1041s)

Even with a suitable base model and carefully prepared data, specialization has limits. Aryan warns against expecting a fine-tuned model to match **GPT-4** on broad, complex tasks merely because it learned the nuances of one dataset. Her reference to **GPT-5** also occurs in this 2023 discussion, not as a measured comparison with that later model. The actionable distinction is between performance inside the target domain and generalization to new data or entirely different domains.

For changing inputs, Aryan proposes **in-context learning with dynamically loaded examples**. The examples can change with the request or current data, providing an inference-time way to address drift while managing costs. She then recommends decomposing broad objectives into smaller tasks: instead of attempting to train for an entire language at once, identify the particular operations the application needs. A narrower task makes both adaptation and evaluation more concrete.

### Trade recomputation for activation memory

[19:02](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1142s)

**Gradient checkpointing** addresses a different part of the training-memory budget. Normally, training retains intermediate activations from the forward pass so the backward pass can calculate gradients. Checkpointing retains fewer intermediates and recomputes the missing ones when needed. Aryan describes this as recomputing weights or retraining; the objects being recomputed are intermediate activations, not a second set of learned weights. The result is additional forward computation in exchange for less activation storage. Whether that trade is worthwhile depends on whether memory or computation is the tighter constraint.

### Treat training settings as hypotheses

[20:00](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1200s)

Aryan suggests batch sizes of 32 or 64 without specifying a model, dataset, or memory budget. She suggests one epoch for a casual Google Colab test and 100 epochs as a starting point for domain optimization. These are the talk's proposed settings, not validated defaults: a useful training budget depends on dataset size, repeated exposure, and held-out performance. A smoke test and a domain-training run answer different questions, so success at the former does not establish the duration of the latter.

Adam is her general-purpose optimizer recommendation. She then turns to regularization and early stopping, arguing that increasing parameter counts can distract from extracting better performance from existing models. Early stopping means monitoring a validation criterion and stopping when further training no longer improves it. If performance declines, Aryan recommends inspecting the batches and embeddings instead of assuming that more training or a larger model is the remedy.

### Evaluate from four perspectives

[21:48](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1308s)

After fine-tuning comes the harder question: did the application improve? Aryan mentions Aviary by Ray and NVIDIA libraries among the available tooling, but emphasizes that loss, accuracy, and perplexity do not provide a complete picture. Evaluation must reflect the business and use case. The historical Aviary repository now leads to the archived [RayLLM repository](https://github.com/ray-project/ray-llm), so that reference should not be treated as a recommendation for a currently maintained evaluation service.

- **Metric-based evaluation:** BLEU and ROUGE provide quantitative comparisons against reference outputs. They offer one perspective on quality rather than a complete judgment of usefulness.
- **Tool-based evaluation:** Aryan mentions Weights & Biases debugging tools and an Auto Evaluator, associating automated checks with catching compilation errors. The precise products and integrations are not specified, so the actionable category is automated detection of concrete failures.
- **Model-based evaluation:** Another model, potentially a smaller one, judges the model being tested. Aryan sees potential for automation and standardization but reports limited success in her own experience.
- **Human-in-the-loop evaluation:** Human review remains a common option, although Aryan considers it inefficient and does not explore it further in this session.

These perspectives expose different kinds of failure. A strong aggregate metric cannot substitute for checking whether the system produces a valid, useful result for its intended users.

### The unit of reliability is the whole pipeline

[23:40](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1420s)

Fine-tuning is only one part of the operating system around a model. Data collection, storage management, and base-model selection constrain what training can achieve. The closing pipeline diagram extends that scope through pretraining, fine-tuning and prompt engineering, evaluation, orchestration, deployment and monitoring, and security and reliability engineering. A one-off demonstration can succeed while leaving several of those responsibilities unresolved.

![“LLMOps Pipeline Components” diagram connecting eight numbered stages: data collection, storage and management; choosing a base model; pre-training; fine-tuning and prompt engineering; evaluation; orchestration; deployment and monitoring; and security and reliability engineering.](https://ai.engineer/api/talk-essay-media/bf2244e8379cf6e10d94509896329339efebb876fef8ca6bbdbe94f6b478168e)

The LLMOps pipeline extends from data collection through deployment, monitoring, security, and reliability.

A durable application must keep working as its data and prompts change. Aryan does not prescribe a fixed testing duration; she focuses instead on detecting problems early enough to avoid exposing an enterprise to reputational or compliance risk. The adaptation decision therefore includes how the system will be monitored, how drift will be recognized, and how failures will be caught—not just which parameters the next training run will update.

## Resources

- [Parameter-Efficient Transfer Learning for NLP](https://arxiv.org/abs/1902.00751) — Original BERT adapter research, with task-specific parameter and GLUE performance results.
- [Prefix-Tuning](https://arxiv.org/abs/2101.00190) — Learned continuous prefixes steer frozen language models for generation and summarization.
- [LoRA: Low-Rank Adaptation of Large Language Models](https://arxiv.org/abs/2106.09685) — The original method for adapting frozen models with trainable low-rank matrices.
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://arxiv.org/abs/2305.14314) — Four-bit frozen model weights combined with LoRA adapters to reduce fine-tuning memory.
- [bitsandbytes](https://github.com/bitsandbytes-foundation/bitsandbytes) — Quantization library with installation guidance and a current hardware support matrix.
- [Deduplicating Training Data Makes Language Models Better](https://arxiv.org/abs/2107.06499) — Experiments connecting training-data duplication, memorized output and evaluation contamination.
- [Training Deep Nets with Sublinear Memory Cost](https://arxiv.org/abs/1604.06174) — Foundational research on trading extra forward computation for lower training memory.

## Slides

### Domain Adaptation and Fine-Tuning for Large Language Models

Abi Aryan introduces an overview of domain adaptation and fine-tuning, aimed at helping readers choose an approach for an enterprise or hobby use case.

[0:19](https://www.youtube.com/watch?v=cXPYtkosXG4&t=19s) [Title slide for Abi Aryan’s AI Engineer Summit presentation on domain adaptation and fine-tuning for large language models.](https://assets.ai.engineer/sites/aiecode2025/sha256/ba67852a6f79c7ea07b8f660b682033ccf277f326c0a413265f75c4b3bac7527.webp)

AI Engineer Summit, 8-10 Oct 2023

DOMAIN ADAPTATION AND FINE-TUNING
FOR LARGE LANGUAGE MODELS

Abi Aryan
October 10, 2023

Interface text:
AIE
AI Engineer SUMMIT
AUTOgpt
smol

### Domain Adaptation: Why do we care?

Domain adaptation aims to improve performance in underrepresented fields such as medicine and law, reduce the need for large newly labeled datasets, and make specialized tasks accessible to more users.

[1:08](https://www.youtube.com/watch?v=cXPYtkosXG4&t=68s) [Slide listing three motivations for domain adaptation: improving performance in specialized domains, reducing data collection and labeling, and broadening access.](https://assets.ai.engineer/sites/aiecode2025/sha256/af489c57d51601323aa6565b1f1258c79b52663d5baede54a9cfc42dca1dc108.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
3

Domain Adaptation: Why do we care?

• To improve the performance of large language models on tasks in underrepresented domains. For example, domain adaptation can be used to improve the performance of large language models on tasks involving medical text, legal documents, or other specialized domains.

• To reduce the need to collect and label large amounts of data for each new domain. Domain adaptation can allow us to use a large language model on a new domain without having to collect and label a large dataset of data from that domain. This can be especially useful for domains where data is scarce or expensive to collect.

• To make large language models more accessible to a wider range of users. Domain adaptation can make it possible to use large language models on tasks in specialized domains, even if the user does not have expertise in that domain.

Abi Aryan | October 10, 2023

### What is Fine-Tuning?

Fine-tuning adapts a pre-trained model to a new task by updating its parameters. The slide pairs a simple network’s weights and biases with a transformer’s encoder and decoder.

[3:38](https://www.youtube.com/watch?v=cXPYtkosXG4&t=218s) [A definition of fine-tuning alongside neural network and transformer diagrams.](https://assets.ai.engineer/sites/aiecode2025/sha256/70fb6ea28994e38c966901aea1ea894462bed9ae35cf44dba19e0b6c178b2a66.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
4

What is Fine-Tuning?

Input layer: 3 × 1
W₁: 5 × 3
Hidden layer: 5 × 1
W₂: 1 × 5
Output layer: 1 × 1

The i-th training example: x⁽ⁱ⁾
b₁: 5 × 1
a₁⁽ⁱ⁾ = σ(W₁x⁽ⁱ⁾ + b₁)
b₂: 1 × 1
The i-th prediction: ŷ⁽ⁱ⁾ = W₂a₁⁽ⁱ⁾ + b₂

Output Probabilities
Softmax
Linear
Encoder N×
Decoder N×
self-attention
encoder-decoder attention
Add & Norm
Feed Forward
Add & Norm
Multi-Head Attention
Add & Norm
Masked Multi-Head Attention
Add & Norm
Feed Forward
Add & Norm
Multi-Head Attention
Positional Encoding
Input Embedding
Inputs
Positional Encoding
Output Embedding
Outputs (shifted right)

Fine-tuning is a technique used in machine learning to improve the performance of pre-trained models on specific tasks.It involves taking a model that has already been trained on a large dataset and retraining it on a new dataset that is related to the original task. Fine-tuning allows data scientists to leverage the knowledge that has already been learned by the model and apply it to a new problem.

Abi Aryan | October 10, 2023

### What is Fine-Tuning?

Full fine-tuning updates all model weights. Partial approaches update some weights to reduce computational and storage costs, though equivalent performance is not guaranteed.

[4:53](https://www.youtube.com/watch?v=cXPYtkosXG4&t=293s) [Full and partial weight updates, with diagrams of distillation and parameter-efficient tuning methods.](https://assets.ai.engineer/sites/aiecode2025/sha256/14e5da5e9291a809b0e9ec0bb31792c3c5f6acfe35c7708076566116dfa7e154.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
5

What is Fine-Tuning?

Updating the model weights
• Update all of the weights (full fine-tuning) - transfer learning, cross-distillation models

Teacher (BERT)
Student (TinyBERT)
Transformer Distillation
Text Input
Transformer Layer:
Embedding Layer:
Prediction Layer:
Layer Number: N > M
Hidden Size: d > d′
N
M
d
d′

• Update some of the weights
- Adaptor Tuning (single adaptor, parallel adaptor, scaled parallel adaptors)
- Prefix Tuning - adds embedding layer at the front of the attention layer to mimic behavior
- Parameter Efficient Fine-Tuning (LoRA, qLoRA)

(a) Adapter
(b) Prefix Tuning
(c) LoRA
(d) Parallel Adapter
(e) Scaled PA
Diagram labels: Add; Gating & Add; Scaling; PLM module; ReLU; Softmax; Wup; Wdown; h; x

- Instruction Tuning
- RLHF

Abi Aryan | October 10, 2023

### How does Fine-tuning work under the hood?

Adapter layers add weights to transformer blocks. The chart shows near-matching accuracy with roughly 150k trainable parameters, compared with 100M for fine-tuning top layers.

[7:20](https://www.youtube.com/watch?v=cXPYtkosXG4&t=440s) [Transformer blocks with and without adapters, alongside a chart comparing accuracy and trainable parameters.](https://assets.ai.engineer/sites/aiecode2025/sha256/02f97d39166f0505aeb7778fbac0b4ebda0b1715903bf798fff9d0ad6e963a41.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
6

How does Fine-tuning work under the hood?

REGULAR TRANSFORMER BLOCK
Multihead self-attention
Fully connected layer
Skip connection
LayerNorm
Fully connected layers
Skip connection
LayerNorm

TRANSFORMER BLOCK WITH ADAPTERS
Multihead self-attention
Fully connected layer
Adapter
Skip connection
LayerNorm
Fully connected layers
Adapter
LayerNorm

ADAPTER LAYERS
Fully connected layer
Nonlinear activation
Fully connected layer

Comparison of the regular transformer blocks used in various LLMs and a transformer block modified via the adapter layers.

Match performance with 0.15% of the parameters
~150k parameters
~100M parameters
Y-axis: Accuracy delta (%)
Y-axis ticks: 5, 0, −5, −10, −15, −20, −25
X-axis: Num trainable parameters / task
X-axis ticks: 10⁵, 10⁶, 10⁷, 10⁸, 10⁹
Legend: Adapters (ours); Fine-tune top layers

Abi Aryan | October 10, 2023

[8:28](https://www.youtube.com/watch?v=cXPYtkosXG4&t=508s) [Transformer blocks with adapters beside a chart comparing trainable parameters and accuracy.](https://assets.ai.engineer/sites/aiecode2025/sha256/a406d1e9e6615e2a90ee29c56a20b97060c88c10cfbc86ab2e8b13f8bade2572.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
6

How does Fine-tuning work under the hood?

REGULAR TRANSFORMER BLOCK
Multihead self-attention
Fully connected layer
Skip connection
LayerNorm
Fully connected layers
Skip connection
LayerNorm

TRANSFORMER BLOCK WITH ADAPTERS
Multihead self-attention
Fully connected layer
Adapter
Skip connection
LayerNorm
Fully connected layers
Adapter
LayerNorm

ADAPTER LAYERS
Fully connected layer
Nonlinear activation
Fully connected layer

Comparison of the regular transformer blocks used in various LLMs and a transformer block modified via the adapter layers.

Match performance with 0.15% of the parameters
~150k parameters
~100M parameters
Y-axis: Accuracy delta (%)
Y-axis ticks: 5, 0, −5, −10, −15, −20, −25
X-axis: Num trainable parameters / task
X-axis ticks: 10⁵, 10⁶, 10⁷, 10⁸, 10⁹
Legend: Adapters (ours); Fine-tune top layers

October 10, 2023

Interface text:
AIE
AI Engineer SUMMIT
Presented by
FIXIE

### Fine-tuning methods compared

Prefix-tuning adds task-specific prefixes to a pretrained transformer. In the five-language chart, ROUGE-2 scores generally rise with training set size; prefix-tuning trails at the largest sizes.

[8:34](https://www.youtube.com/watch?v=cXPYtkosXG4&t=514s) [Diagrams of fine-tuning, prefix-tuning and low-rank adaptation alongside a performance chart.](https://assets.ai.engineer/sites/aiecode2025/sha256/85b467b1a88de5b9c87ee7922722496a58abf200f62363959635a74e1147f74b.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
7

How does Fine-tuning work under the hood?

Fine-tuning
Transformer (Translation)
Transformer (Summarization)
Transformer (Table-to-text)
name Starbucks type coffee shop [SEP] Starbucks serves coffee
Input (table-to-text)  Output (table-to-text)

Prefix-tuning
Prefix (Translation)
Prefix (Summarization)
Prefix (Table-to-text)
Transformer (Pretrained)
name Starbucks type coffee shop [SEP] Starbucks serves coffee
Input (table-to-text)  Output (table-to-text)

Q  K  V
h
Pretrained Weights
W ∈ Rᵈˣᵈ
B = 0
r
A = N(0, σ²)
d
x
h = W₀x + ΔWx = W₀x + BAx

Chart legend: prefix-tuning; adapter-tuning; PLM fine-tuning
Y-axis: R2
Y-axis ticks: 1, 2, 3, 4, 5, 6, 7, 8, 9
X-axis: training set size
X-axis ticks: 10¹, 10², 10³, 10⁴

Performance of prefix-tuning, adapter-tuning and PLM fine-tuning on five languages over training set sizes. The x axis is the number of training samples at log scale, the y axis is the ROUGE-2 score.

October 10, 2023

AIE
AI Engineer SUMMIT
Presented by
AUTOgpt
smol ai

### Fine-tuning comparison detail

Prefix-tuning adds task-specific prefixes to a shared pretrained transformer. ROUGE-2 generally rises with training set size; adapter-tuning and PLM fine-tuning finish above prefix-tuning.

[8:37](https://www.youtube.com/watch?v=cXPYtkosXG4&t=517s) [Enlarged fine-tuning diagrams and a chart comparing prefix-tuning, adapter-tuning and model fine-tuning across training set sizes.](https://assets.ai.engineer/sites/aiecode2025/sha256/50645d148c57903caa93e02aa7fa248e3cfc510152ad51094e86f9f8b4481c38.webp)

Fine-tuning
Transformer (Translation)
Transformer (Summarization)
Transformer (Table-to-text)
name Starbucks type coffee shop [SEP] Starbucks serves coffee
Input (table-to-text) Output (table-to-text)

Prefix-tuning
Prefix (Translation)
Prefix (Summarization)
Prefix (Table-to-text)
Transformer (Pretrained)
name Starbucks type coffee shop [SEP] Starbucks serves coffee
Input (table-to-text) Output (table-to-text)

Q K V
h
Pretrained Weights
W ∈ ℝᵈˣᵈ
B = 0
r
A = 𝒩(0, σ²)
d
x
h = W₀x + ΔWx = W₀x + BAx

Chart legend: prefix-tuning; adapter-tuning; PLM fine-tuning
Y-axis: R2
Y-axis ticks: 1, 2, 3, 4, 5, 6, 7, 8, 9
X-axis: training set size
X-axis ticks: 10¹, 10², 10³, 10⁴

Performance of prefix-tuning, adapter-tuning and PLM fine-tuning on five languages over training set sizes. The x axis is the number of training samples at log scale, the y axis is the ROUGE-2 score.

### How does Fine-tuning work under the hood?

LoRA adds a low-rank A–B update to pretrained weights. For GPT-3 175B, the slide reports training VRAM falling from 1.2TB to 350GB; with r = 4 and only query and value projections adapted, checkpoints fall from 350GB to 35MB.

[9:48](https://www.youtube.com/watch?v=cXPYtkosXG4&t=588s) [Pretrained weights and low-rank matrices feed an output, beside an excerpt about training memory and checkpoint size.](https://assets.ai.engineer/sites/aiecode2025/sha256/4484cbf280dfa795599502e91582d6318e59af397a3031aa4f91a99ee3dc458c.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
8

How does Fine-tuning work under the hood?

h
+
Pretrained Weights
W ∈ ℝᵈˣᵈ
B = 0
r
A = 𝒩(0, σ²)
d
x

Practical Benefits and Limitations. The most significant benefit comes from the reduction in memory and storage usage. For a large Transformer trained with Adam, we reduce that VRAM usage by up to 2/3 if r ≪ d_model as we do not need to store the optimizer states for the frozen parameters. On GPT-3 175B, we reduce the VRAM consumption during training from 1.2TB to 350GB. With r = 4 and only the query and value projection matrices being adapted, the checkpoint size is reduced by roughly 10,000× (from 350GB to 35MB)⁴. This allows us to train with signifi-

Abi Aryan | October 10, 2023

[11:01](https://www.youtube.com/watch?v=cXPYtkosXG4&t=661s) [Diagram of pretrained weights and low-rank adaptation matrices, alongside an excerpt about memory and storage savings.](https://assets.ai.engineer/sites/aiecode2025/sha256/5cef74b9bffee0538a70017185871f559190cf7ee831c930b459edfc5eea9a29.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
8

How does Fine-tuning work under the hood?

h
+
Pretrained Weights
W ∈ ℝᵈˣᵈ
B = 0
r
A = 𝒩(0, σ²)
d
x

Practical Benefits and Limitations. The most significant benefit comes from the reduction in memory and storage usage. For a large Transformer trained with Adam, we reduce that VRAM usage by up to 2/3 if r ≪ d_model as we do not need to store the optimizer states for the frozen parameters. On GPT-3 175B, we reduce the VRAM consumption during training from 1.2TB to 350GB. With r = 4 and only the query and value projection matrices being adapted, the checkpoint size is reduced by roughly 10,000× (from 350GB to 35MB)⁴. This allows us to train with signifi-

October 10, 2023

Interface text:
AIE
AI Engineer SUMMIT
Presented by
FIXIE
S

### Full Fine-Tuning, LoRA and QLoRA

QLoRA uses a 4-bit transformer with adapters and CPU paging. On LLaMA 7B’s Alpaca RougeL evaluation, QLoRA on all layers scores near the 16-bit baseline; FFN-only and attention-only variants score lower.

[12:14](https://www.youtube.com/watch?v=cXPYtkosXG4&t=734s) [Diagram comparing full fine-tuning, LoRA and QLoRA, with a chart of model performance on the Alpaca dataset.](https://assets.ai.engineer/sites/aiecode2025/sha256/f82e3b270b4c85abd32b8e1bd2ed38c91b2703076525633d75d4c10d5efde608.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
9

How does Fine-tuning work under the hood?

Full Finetuning (No Adapters) | LoRA | QLoRA
Optimizer State (32 bit)
Adapters (16 bit)
Base Model
16-bit Transformer | 16-bit Transformer | 4-bit Transformer
CPU
Parameter Updates
Gradient Flow
Paging Flow

Chart:
Y-axis: RougeL
Y-axis ticks: 60, 61, 62, 63, 64
X-axis: Model
Models: QLoRA-All, QLoRA-FFN, QLoRA-Attention, Alpaca (ours), Stanford Alpaca
Legend: bits — 4, 16

Figure 2: RougeL for LLaMA 7B models on the Alpaca dataset. Each point represents a run with a different random seed. We improve on the Stanford Alpaca fully finetuned default hyperparameters to construct a strong 16-bit baseline for comparisons. Using LoRA on all transformer layers is critical to match 16-bit performance.

Abi Aryan | October 10, 2023

### Domain Adaptation: Fine-Tuning vs RAGs vs Prompting

RAG retrieves relevant information; adaptive fine-tuning targets a domain, while behavioral fine-tuning targets a task. Aryan says these approaches can be combined with prompting.

[13:41](https://www.youtube.com/watch?v=cXPYtkosXG4&t=821s) [Three columns compare prompting, retrieval augmented generation and fine-tuning for domain adaptation.](https://assets.ai.engineer/sites/aiecode2025/sha256/51edd2595c1b3a2fbb5502973153fdaba26a9660698eb8af4f61a6955b45ac5f.webp)

Domain Adaptation: Fine-Tuning vs RAGs vs Prompting

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
10

Prompting

Prompt Engineering is for demos. Low implementation complexity.

• Zero-Shot - the chat-prompting
• One-Shot - give an adaptive example
• Few-Shot - give couple of examples

Prompt Tuning - introduces additional parameters in the prompts and optimizes those parameters with supervised examples

Pros:
• more generalizable models
• requires less training data
• cheaper since the cost is literally inference cost only

Retrieval Augmented Generation (RAG)

Information retrieval relevant to a task usually stored in vector stores

Uses embedding model to vectorize input query and then uses semantic similarity to measure the distance between input vector and document vectors

• most useful for updating the model with factual information
• reduce hallucination

Fine-Tuning

Fine-tuning is suitable for continuous domain adaptation

Adaptive Fine-Tuning- given the target domain you are learning the feature space and the distribution of target data (most useful when you are training for multiple tasks in a single domain for very high accuracy

Behavioral Fine-Tuning - for target task, you are learning the label space and prior PE distribution, most useful when you are looking for high level inference and reasoning capability on a task

Parameter Efficient Fine-Tuning - updating only some of the weights and interpolating them - optimizing resources like memory and space - lower performance but high speed

October 10, 2023

Interface text:
AIE
AI Engineer SUMMIT
Presented by
AUTOGPT
smol ai

[13:48](https://www.youtube.com/watch?v=cXPYtkosXG4&t=828s) [Three columns compare prompting, retrieval augmented generation, and fine-tuning.](https://assets.ai.engineer/sites/aiecode2025/sha256/ad1677345a8abd90ed9bcfcb78d97ac905cf3089620b91c8f0e78c8dad21ab7a.webp)

Domain Adaptation: Fine-Tuning vs RAGs vs Prompting

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
10

Prompting

Prompt Engineering is for demos. Low implementation complexity.

• Zero-Shot - the chat-prompting
• One-Shot - give an adaptive example
• Few-Shot - give couple of examples

Prompt Tuning - introduces additional parameters in the prompts and optimizes those parameters with supervised examples

Pros:
• more generalizable models
• requires less training data
• cheaper since the cost is literally inference cost only

Retrieval Augmented Generation (RAG)

Information retrieval relevant to a task usually stored in vector stores

Uses embedding model to vectorize input query and then uses semantic similarity to measure the distance between input vector and document vectors

• most useful for updating the model with factual information
• reduce hallucination

Fine-Tuning

Fine-tuning is suitable for continuous domain adaptation

Adaptive Fine-Tuning- given the target domain you are learning the feature space and the distribution of target data (most useful when you are training for multiple tasks in a single domain for very high accuracy

Behavioral Fine-Tuning - for target task, you are learning the label space and prior PE distribution, most useful when you are looking for high level inference and reasoning capability on a task

Parameter Efficient Fine-Tuning - updating only some of the weights and interpolating them - optimizing resources like memory and space - lower performance but high speed

Abi Aryan | October 10, 2023

### Your Fine-Tuned Models Are Only as Good as Your Data

Data preparation spans collection, tokenization, cleaning, noise removal, deduplication, and augmentation. Aryan notes that removing duplicates can reduce memorization and bias toward repeated content.

[16:18](https://www.youtube.com/watch?v=cXPYtkosXG4&t=978s) [A list covers data collection, tokenization, cleaning, noise removal, deduplication, and augmentation.](https://assets.ai.engineer/sites/aiecode2025/sha256/fc018272661486d2bf27e5d213cb5e15f9d80e069b5cd751534bc20e16ee203b.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
11

Your fine-tuned models are only as good as your data

• Data Collection and Aggregation

• Text Tokenization:
The collected text is tokenized into smaller units, typically words or subword units. Tokenization helps break down the text into manageable chunks, making it easier for the model to process.

• Text Cleaning and Normalisation:
This step involves removing any unnecessary characters, formatting, or special symbols that might not contribute to the model's learning. It also includes tasks like converting text to lowercase, fixing typos, and handling special cases like contractions.

• Removing Noise:
Irrelevant or noisy data, such as advertisements, boilerplate text, or user-generated noise, is filtered out during this stage. This helps maintain the quality and relevance of the training data.

• Deduplication:
Duplicate entries are identified and removed to prevent the model from being biased towards repeated content.

• Data Augmentation:
In some cases, data augmentation techniques are applied to increase the diversity of the training data. This can involve tasks like synonym replacement, paraphrasing, or other linguistic transformations.

Abi Aryan | October 10, 2023

### Key Considerations and Limitations

Aryan cautions that weaker models can struggle with complex tasks. She suggests dynamic in-context examples to address data drift and costs, and smaller subtasks to expose bottlenecks.

[17:23](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1043s) [The slide discusses model imitation, in-context learning with dynamic examples, and breaking tasks into subtasks.](https://assets.ai.engineer/sites/aiecode2025/sha256/200030fff6fa7e2607869f4a5727262d337d7908c069be99aad74cfa44e9e6b9.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
12

Key Considerations and Limitations

To improve the performance of a weaker model, you can:

• Avoid imitating the outputs of more sophisticated models. This can lead to poor performance, especially for complex tasks.

This is because weaker models may not be able to capture all of the nuances of the training data. As a result, they may learn to produce outputs that are similar to those of the more sophisticated models, but not necessarily correct.

• Use in-context learning with dynamic example loading. This can achieve the same results as fine-tuning, but without the additional costs of a managed service.

This involves training the model on a small set of examples that are dynamically loaded from the training data. This approach can be effective because it allows the model to learn from new examples at runtime, without having to retrain the entire model. This also helps with model drift.

• Break down large tasks into smaller subtasks. This can make them more manageable and help to identify and address bottlenecks related to model limitations.

For example, if you are training a model to translate text from one language to another, you could break down the task into smaller subtasks, such as translating individual words and phrases. This can help you to identify areas where the model is struggling and to develop strategies for improving its performance.

Abi Aryan | October 10, 2023

### Key Considerations and Limitations: Gradient Checkpointing

Gradient checkpointing saves GPU memory by clearing selected activations and recomputing them during the backward pass. The tradeoff is more computation.

[19:07](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1147s) [The slide describes reducing memory use by recomputing activations during the backward pass.](https://assets.ai.engineer/sites/aiecode2025/sha256/508fb75fe3700c7b6446b521818806f7287ce8e0b34abaa2ae18a864036820ed.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
13

Key Considerations and Limitations

To improve the performance of a weaker model, you can:

• Implement Gradient checkpointing

LLMs have a large number of parameters and activations, which can quickly consume all of the available GPU memory. Gradient checkpointing reduces the memory usage by clearing the activations of certain layers and recomputing them during the backward pass. This works by dividing the forward pass of the model into multiple chunks. For each chunk, the activations are computed and stored in memory. Once the activations for the entire model have been computed, the backward pass is performed. During the backward pass, the activations for each chunk are recomputed as needed.

Abi Aryan | October 10, 2023

[19:48](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1188s) [Slide explaining how gradient checkpointing reduces GPU memory use by recomputing activations during the backward pass.](https://assets.ai.engineer/sites/aiecode2025/sha256/d7624d2760bcc6fd35dfe53c1cd35bb919618614dd94f16b9ad7e55e8bcc027d.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
13

Key Considerations and Limitations

To improve the performance of a weaker model, you can:

• Implement Gradient checkpointing

LLMs have a large number of parameters and activations, which can quickly consume all of the available GPU memory. Gradient checkpointing reduces the memory usage by clearing the activations of certain layers and recomputing them during the backward pass. This works by dividing the forward pass of the model into multiple chunks. For each chunk, the activations are computed and stored in memory. Once the activations for the entire model have been computed, the backward pass is performed. During the backward pass, the activations for each chunk are recomputed as needed.

October 10, 2023

Interface text:
AIE
AI Engineer SUMMIT
AUTOgpt
smol ai

### Training considerations

Batch size and learning rate trade convergence speed against memory or stability. Aryan suggests 32 or 64 for batch size and 100 epochs as a starting point, while monitoring validation loss for overfitting.

[20:05](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1205s) [Slide covering batch size, epochs, optimizer, learning rate, regularization and early stopping.](https://assets.ai.engineer/sites/aiecode2025/sha256/5ce32aaa90c760d869829675637a0821f2b8fb84d85b4bf599ae4739805b3ba1.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
14

Key Considerations and Limitations

• Choosing a batch size
Larger batch sizes can lead to faster convergence, but they can also require more memory and can be more sensitive to noise in the training data. Smaller batch sizes can be more robust to noise, but they can also lead to slower convergence.A good starting point for choosing a batch size for LLMs is to use a batch size of 32 or 64. If you have a large dataset and a powerful GPU, you can try using a larger batch size. However, if you have a limited amount of memory or your GPU is not very powerful, you may need to use a smaller batch size.

• Choosing the number of training epochs
More training epochs can lead to better performance, but they can also lead to overfitting. A good starting point for choosing the number of training epochs for LLMs is to use 100 epochs. If you are not seeing any improvement in performance after 100 epochs, you can try training for longer. However, it is important to monitor the validation loss to make sure that the model is not overfitting.

• Choosing an optimizer
There are many different optimizers available, each with its own strengths and weaknesses.A good choice for training LLMs is the Adam optimizer. Adam is a general-purpose optimizer that has been shown to work well for a wide variety of machine learning tasks. It is also relatively easy to tune.

• Learning rate: The learning rate controls how quickly the model parameters are updated. A higher learning rate can lead to faster convergence, but it can also make the model more unstable. A lower learning rate can lead to slower convergence, but it can also make the model more stable.

• Regularization: Regularization techniques can help to prevent overfitting. Some common regularization techniques for LLMs include dropout and weight decay.

• Early stopping: Early stopping is a technique that can be used to stop training the model early if it is overfitting.

Abi Aryan | October 10, 2023

### Evaluation metrics and benchmarks

Loss, accuracy and perplexity do not paint the full picture. The slide compares benchmarks for general language tasks and specific downstream tasks, with criteria ranging from legal understanding to code generation.

[21:51](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1311s) [Slide listing loss, accuracy and perplexity beside a table of evaluation benchmarks, domains and criteria.](https://assets.ai.engineer/sites/aiecode2025/sha256/734c0665fb6ff6ffb2def42253fe8d1a7f2d77ae1fa0ccd40b9dbfba8431d5b4.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS
15

Evaluation is likely the hardest part.. What to measure

□ Loss
□ Accuracy
□ Perplexity

Benchmark | Focus | Domain | Evaluation Criteria
SOCKET (Choi et al., 2023) | Social knowledge | Specific downstream task | Social language understanding
MME (Fu et al., 2023a) | Multimodal LLMs | General language task | Ability of perception and cognition
Xiezhi (Gu et al., 2023) | Comprehensive domain knowledge | General language task | Overall performance across multiple benchmarks
CUAD (Hendrycks et al., 2021b) | Legal contract review | Specific downstream task | Legal contract understanding
TRUSTGPT (Huang et al., 2023c) | Ethic | Specific downstream task | Toxicity, bias, and value-alignment
MMLU (Hendrycks et al., 2020b) | Text models | General language task | Multitask accuracy
MATH (Hendrycks et al., 2021c) | Mathematical problem | Specific downstream task | Mathematical ability
APPS (Hendrycks et al., 2021a) | Coding challenge competence | Specific downstream task | Code generation ability
C-Eval (Huang et al., 2023b) | Chinese evaluation | General language task | 52 Exams in a Chinese context
OpenLLM (HuggingFace, 2023) | Chatbots | General language task | Leaderboard rankings
DynaBench (Kiela et al., 2021) | Dynamic evaluation | General language task | NLI, QA, sentiment, and hate speech
Chatbot Arena (LMSYS, 2023) | Chat assistants | General language task | Crowdsourcing and Elo rating system
AlpacaEval (Li et al., 2023d) | Automated evaluation | General language task | Metrics, robustness, and diversity
HELM (Liang et al., 2022) | Transparency of language models | General language task | Multi-metric
API-Bank (Li et al., 2023a) | Tool utilization | Specific downstream task | API call, retrieval, and planning
M3KE (Liu et al., 2023a) | Multi-task | General language task | Multi-task accuracy
ARB (Sawada et al., 2023) | Advanced reasoning ability | Specific downstream task | Multidomain advanced reasoning ability
Big-Bench (Srivastava et al., 2022) | Capabilities and limitations of LMs | General language task | Model performance and calibration
MultiMedQA (Singhal et al., 2022) | Medical QA | Specific downstream task | Model performance, medical knowledge, and reasoning ability
CVALUES (Xu et al., 2023b) | Safety and responsibility | Specific downstream task | Alignment ability of LLMs
ToolBench (ToolBench, 2023) | Software tools | Specific downstream task | Execution success rate
PandaLM (Wang et al., 2023h) | Instruction tuning | General language task | Winrate judged by PandaLM
GLUE-X (Yang et al., 2022) | OOD robustness for NLU tasks | General language task | OOD robustness
KoLA (Yu et al., 2023) | Knowledge-oriented evaluation | General language task | Self-contrast metrics
AGIEval (Zhong et al., 2023) | Human-centered foundational models | General language task | General
PromptBench (Zhu et al., 2023) | Adversarial prompt resilience | General language task | Adversarial robustness
MT-Bench (Zheng et al., 2023) | Multi-turn conversation | General language task | Winrate judged by GPT-4
M3Exam (Zhang et al., 2023c) | Human exams | Specific downstream task | Task-specific metrics
GAOKAO-Bench (Zhang et al., 2023e) | Chinese Gaokao examination | Specific downstream task | Accuracy and scoring rate

Abi Aryan | October 10, 2023

### Thank you

[24:46](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1486s) [Closing slide reading “Thank You!”](https://assets.ai.engineer/sites/aiecode2025/sha256/74364d4853a5f7a6d3a387576ddeec2e3977bf19bb245d5271a6e24cac781048.webp)

DOMAIN ADAPTATION AND FINE-TUNING FOR LARGE LANGUAGE MODELS

Thank You!

18
October 10, 2023

AIE
FIXIE
supabase
AI Engineer SUMMIT

## Full transcript

[0:00](https://www.youtube.com/watch?v=cXPYtkosXG4&t=0s) [on-hold electronic music] Hi, everyone.

[0:16](https://www.youtube.com/watch?v=cXPYtkosXG4&t=16s) Welcome to my presentation. So I don't know what's the best way to start about it. I would probably say something along the lines of, well, you've heard a lot of really good presentation that have focused on one very specific thing.

[0:32](https://www.youtube.com/watch?v=cXPYtkosXG4&t=32s) In this session itself, we'll more so focus on an overview of domain adaptation and fine-tuning for large language models. Because there's so much information out there which is like, "Oh, take this, use that."

[0:47](https://www.youtube.com/watch?v=cXPYtkosXG4&t=47s) So my goal is to sum up all the literature for you to be able to make an informed decision on how to be able to do domain adaptation for your particular enterprise use case or for your hobby use case, however you're using it for.

[1:02](https://www.youtube.com/watch?v=cXPYtkosXG4&t=62s) So about me, uh, this has already [chuckles] been spoken, so let's, let's skip this. Um, why do we care? I think, I think the answer to this is pretty obvious, which is, I mean, there are-- ChatGPT as a model, or even if you're looking at open source large language models, they're not trained for every single use case out

[1:23](https://www.youtube.com/watch?v=cXPYtkosXG4&t=83s) there. There are some domains that are underrepresented. There are some domains for which there is not enough data because of compliance or for whatever reasons. And for that, we need to be able to have some sort of method to be able to fine-tune the models or use some other strategy and alternative of fine-tuning, whether that's knowledge bases,

[1:44](https://www.youtube.com/watch?v=cXPYtkosXG4&t=104s) whether that's RAGs, or whether that's prompting. The second is basically you don't want to collect new data for every single domain. One of the best things that has happened with large language models, I would say, is the ability of these models to be able to transition to a new domain.

[2:02](https://www.youtube.com/watch?v=cXPYtkosXG4&t=122s) So there's one paper that I would reference. Uh, so one quick example I would say is before, let's say, transformer models, uh, or even while we were having transformer models, um, to be able to train a model to learn a new language, we needed to collect the data for that particular language, and then

[2:25](https://www.youtube.com/watch?v=cXPYtkosXG4&t=145s) to be able to, um, do whatever task that we want to do in that language. One of the best things that has happened is now, because the models are learning via embeddings, they're able to learn on a new language that they've previously not seen as well, because they're essentially learning the structure of the languages instead of like

[2:46](https://www.youtube.com/watch?v=cXPYtkosXG4&t=166s) what is the taxonomy of the language. Which means there are some languages which are semantically similar. So for example, English is very semantically similar to Latin. I'm not entirely sure there are a couple of languages that do fall into like that one domain, which is, oh, these languages are similar.

[3:05](https://www.youtube.com/watch?v=cXPYtkosXG4&t=185s) They have semantic similarities. There are other set of languages that have semantic similarities. So it's very easy to be able to transition between those languages without ever having seen any data or any examples in those languages.

[3:18](https://www.youtube.com/watch?v=cXPYtkosXG4&t=198s) The third is basically you want the models to be able to be accessible to a wide range of users. And what I mean by that is more so like all the work that was happening along in personalization.

[3:32](https://www.youtube.com/watch?v=cXPYtkosXG4&t=212s) So simple reasons. This is something alr- almost everybody is aware of. What is fine-tuning? Fine-tuning is almost a way of us,

[3:43](https://www.youtube.com/watch?v=cXPYtkosXG4&t=223s) um, teaching the model to be able to learn something for which it hasn't already been trained before, so improving the performance of a pre-trained model. Um, one of the ways we're doing that is by updating the parameters, right?

[4:02](https://www.youtube.com/watch?v=cXPYtkosXG4&t=242s) You take some input, you have a hidden layer, um, in which you're calculating the weights, you're calculating the biases, and then you have an output layer. That all stuff I think is obvious to almost everybody.

[4:14](https://www.youtube.com/watch?v=cXPYtkosXG4&t=254s) You've seen what a transformer model. For people who don't know what the structure of transformer model is, there's an encoder, there's a decoder. The reason I'm referencing this is we'll go a little bit more into details of these while we are talking about different fine-tuning methods itself.

[4:30](https://www.youtube.com/watch?v=cXPYtkosXG4&t=270s) So there's an encoder, there's a decoder. It has, uh, s- a feed forward network. It has an attention network. Same for the decoder one. Now, this is, this is how we were looking at transformer models they are-- the way they are, and this is storing the weights and the biases right now.

[4:51](https://www.youtube.com/watch?v=cXPYtkosXG4&t=291s) But now let's talk about making these models better. So there are a couple of ways that we can fine-tune our methods. We can update all the model weights, or we can update some of the weights.

[5:05](https://www.youtube.com/watch?v=cXPYtkosXG4&t=305s) If we update all the model weights, that falls into the category of some of the models that you've seen earlier, which is the all the research work that was between two thousand and eighteen, two thousand sixteen, all those, all those years, which is more around transfer learning cross distillation models in which you have a teacher model and

[5:23](https://www.youtube.com/watch?v=cXPYtkosXG4&t=323s) a student model. The student model is learning from the teacher model, and that's, that's the way you're sort of updating all the weights. But it is very expensive to do that, and it is computationally--

[5:36](https://www.youtube.com/watch?v=cXPYtkosXG4&t=336s) It takes more storage as well. The second option that we've-- we're now looking at, the reason we are having this discussion today is f- how can we update our models?

[5:48](https://www.youtube.com/watch?v=cXPYtkosXG4&t=348s) Because the parameters have gone s- so big, we cannot keep updating [chuckles] all the weights. So how about we update just some of the weights without making sure, uh, while making sure that we're able to get equivalent performance and-

[6:05](https://www.youtube.com/watch?v=cXPYtkosXG4&t=365s) I would, I would put an asterisk on, you know, like equivalent performance because we may not be able to get the ChatGPT performance, and that is something we'll talk about eventually.

[6:16](https://www.youtube.com/watch?v=cXPYtkosXG4&t=376s) So in terms of if we update some of the weights, you can break it down into three categories. To be honest, more like five categories, but there are three main ones, which is adapter tuning, uh, there's prefix tuning, and there's parameter-efficient tuning.

[6:30](https://www.youtube.com/watch?v=cXPYtkosXG4&t=390s) There's instruction tuning, which is basically giving a couple of examples. Um, this is something you've seen a lot at couple of-- so many examples throughout this conference, and the one that was prior to my talk as well, uh, where we were doing instruction tuning.

[6:45](https://www.youtube.com/watch?v=cXPYtkosXG4&t=405s) RLHF, obviously not super relevant to most of us, which is we would-- it's too expensive to have real human beings to be able to fine-tune your parameters for you, um, or to be able to provide your examples and say, "This is wrong.

[7:01](https://www.youtube.com/watch?v=cXPYtkosXG4&t=421s) This is right." So we are only left with three techniques, which is adapter tuning, prefix tuning, and fine-- uh, parameter-efficient fine-tuning. We'll go a little bit more into detail of what these are, why are we using these ones,

[7:17](https://www.youtube.com/watch?v=cXPYtkosXG4&t=437s) and when do they do well. So the first one, this is basically adapter-based tuning. The, the thing that really happens in adapter-based tuning is it's really good. Uh, what it does is it adds a small number of parameters to the existing model.

[7:34](https://www.youtube.com/watch?v=cXPYtkosXG4&t=454s) Uh, those parameters are basically stored in the adapter components that you're seeing over there. This is the entire model of the transformer remains the same, but we are adding two new components to it that contains the extra weights.

[7:49](https://www.youtube.com/watch?v=cXPYtkosXG4&t=469s) So what this does is this exposes the model to the new information, and according to, according to the original paper that came out, you know, it is able to improve the performance of the model.

[8:02](https://www.youtube.com/watch?v=cXPYtkosXG4&t=482s) Or you could say it matches the performance of the model with only zero point fifteen percent of the parameters. Where is it good? Where-- In which cases would we use something like this?

[8:14](https://www.youtube.com/watch?v=cXPYtkosXG4&t=494s) So adapter fine-tuning or adaptive fine-tuning, both are the same things. Um, ideally, you use it when you're trying to learn a new domain itself, which is if you're trying to fine-tune your model for like a very different domain, let's say biochemical engineering, that's, that's more so where you would use, uh, adapter-based fine-tuning.

[8:36](https://www.youtube.com/watch?v=cXPYtkosXG4&t=516s) The second is prefix-based fine-tuning. So prefix-based fine-tuning, what it does is it introduces some prefixes where we are storing the model weights, and what they're able to do is they're able to mimic the behavior of the prefix that we are giving it, which is the couple of weights that we are adding in front of that attention model.

[8:58](https://www.youtube.com/watch?v=cXPYtkosXG4&t=538s) Um, so in a very simple-- in very simple words, what it does, it, it, um, it adds an embedding layer at the front of the attention layer, uh, to mimic that behavior.

[9:11](https://www.youtube.com/watch?v=cXPYtkosXG4&t=551s) One very simple example to understand this a little bit better is, you know, all of the water that we get comes out of a tank, right? But the way we are able to access it is using a tap, and water takes the form of a tap, which is it comes out in, in this quantity.

[9:30](https://www.youtube.com/watch?v=cXPYtkosXG4&t=570s) So that's, that's very much like how prefix tuning works, which is it's not changing the behavior of the model, but it's just mimicking or adding a masking layer on top of, uh, the existing weights or on top of the existing model that there is.

[9:46](https://www.youtube.com/watch?v=cXPYtkosXG4&t=586s) The third and the final one, which is the prefix ba-- uh, which is the parameter-efficient fine-tuning method. So this one, the one example that you're seeing is basically the LoRA one.

[9:57](https://www.youtube.com/watch?v=cXPYtkosXG4&t=597s) There are two commonly known parameter-efficient fine-tuning methods that are out there, LoRA and QLoRA. The way LoRA really works, uh, what it really is, is basically Low-Rank Adaptation method.

[10:11](https://www.youtube.com/watch?v=cXPYtkosXG4&t=611s) Where you use it, any sort of parameter-efficient fine-tuning method is sort of used where you want to compress the model sizes, or you want to run it on low-resource devices.

[10:22](https://www.youtube.com/watch?v=cXPYtkosXG4&t=622s) So very ideal for large language models. Biggest reason is because we have massive parameters that we are trying to run on very small devices, which could be our laptops and even smaller devices, which is basically the HTML devices, the Arduino, and all of that stuff.

[10:39](https://www.youtube.com/watch?v=cXPYtkosXG4&t=639s) So that's one reason the entire community has been talking more so about LoRA and QLoRA because, again, we are looking for efficiency. The way it works under the hood is, um, all of the weights are usually stored as what is basically a matrix, right?

[10:56](https://www.youtube.com/watch?v=cXPYtkosXG4&t=656s) So most of these weights, um, there, there are a lot of layers in these weights that aren't, uh, unique. And what I would-- what LoRA usually does is it identifies the linearly independent layers, um, in, in terms of the weight, uh, matrix itself.

[11:18](https://www.youtube.com/watch?v=cXPYtkosXG4&t=678s) So in the matrix, you're looking at all the linearly independent lines or the columns, and you're picking and choosing only those ones. So what it does is if two things are very similar or two things are almost like you could transform one easily through a mathematical function as like a multiplier of the other one, then storing that

[11:40](https://www.youtube.com/watch?v=cXPYtkosXG4&t=700s) one extra layer, which is a copy of the original one, doesn't really make sense, right? So that's, that's how LoRA works under the hood, which is we are reducing the size of the matrix, which is the size of the weight matrix, essentially.

[11:54](https://www.youtube.com/watch?v=cXPYtkosXG4&t=714s) Practical benefits, obviously, um, you know, you're, you're able to decrease the size of the model, and you're also-- you can also, um- You can also-- You're, you're also using less memory right now.

[12:11](https://www.youtube.com/watch?v=cXPYtkosXG4&t=731s) Um, the second method that we are looking at is basically called QLoRA, which is quantized, uh, LoRA method. The way it works is it changes the model weights to four-bit precision.

[12:25](https://www.youtube.com/watch?v=cXPYtkosXG4&t=745s) Um, the way it usually works is you start with the pre-trained models. Uh, you collect a dataset with labeled, um, with labeled data, and you train adaptation matrix, uh, and multiply it with the main weight matrix.

[12:42](https://www.youtube.com/watch?v=cXPYtkosXG4&t=762s) And what you're essentially trying to do is you're trying to decrease the distance between the predicted outputs of the source domain and the target domain. That's, that's what's essentially going on in QLoRA.

[12:56](https://www.youtube.com/watch?v=cXPYtkosXG4&t=776s) One quick comparison, uh, obviously, I mean, w- w- in, in terms of like the people who are saying, "Okay, QLoRA is great. Should we use LoRA, QLoRA?" Um,

[13:08](https://www.youtube.com/watch?v=cXPYtkosXG4&t=788s) one quick thing I'll say on that one is while QLoRA works really good on the original dataset that it was trained on, but to be able to get it to perform really well requires a library, bitsandbytes libraries, and some other things which are not available on all the devices.

[13:26](https://www.youtube.com/watch?v=cXPYtkosXG4&t=806s) Not a lot of testing has really happened for QLoRA's efficiency on all of the models. So I would probably say maybe still sticking with LoRA and being able to optimize the performance with the LoRA model is ideally like the better way to go, at least at, uh, this current point in time.

[13:43](https://www.youtube.com/watch?v=cXPYtkosXG4&t=823s) So to, to very quickly summarize, um, which is, again, we have three different methods to be able to do d- domain adaptation. We have prompting, we have RAGs, we have fine-tuning.

[13:57](https://www.youtube.com/watch?v=cXPYtkosXG4&t=837s) For prompting, you can sort of prompt your models, again, with no examples, with one example, with a couple of examples. When it comes to a couple of examples, I think a good answer would be about ten, which is what ChatGPT says, where obviously the performance is better the more examples you're able to give it.

[14:15](https://www.youtube.com/watch?v=cXPYtkosXG4&t=855s) Where it works is, um, in, in the domains that you're looking for more generalizable models, but usually that's just demos, not real-world examples. Requires less training data. It's cheaper obviously, but is not as performant as fine-tuning.

[14:30](https://www.youtube.com/watch?v=cXPYtkosXG4&t=870s) On fine-tuning, you're looking at three different methods, which is like adaptive fine-tuning, you're looking at behavioral fine-tuning, and parameter-efficient fine-tuning. Um, on each of these ones, um, you don't need to pick one of these three techniques.

[14:46](https://www.youtube.com/watch?v=cXPYtkosXG4&t=886s) You can also combine them with prompt engineering. You can combine them with RAGs as well, or you can do both of those things, which is you can do adaptive as well as behavioral fine-tuning.

[14:56](https://www.youtube.com/watch?v=cXPYtkosXG4&t=896s) The key difference between all those three methods is adaptive fine-tuning really works well on when you have a target domain that you're trying to optimize for. So for example, like if you have multiple tasks within a single domain, let's say you are, um, you have a legal company, and you're trying to build a model that works really

[15:16](https://www.youtube.com/watch?v=cXPYtkosXG4&t=916s) well on five different or ten different tasks within just the legal domain itself, adaptive fine-tuning works great. Behavioral fine-tuning is basically where you're trying to optimize the model performance on a target task only.

[15:29](https://www.youtube.com/watch?v=cXPYtkosXG4&t=929s) So you're not really optimizing for the entire domain, you're optimizing for just one particular task. The way it really works is you're optimizing for the label space and the prior probability distribution.

[15:43](https://www.youtube.com/watch?v=cXPYtkosXG4&t=943s) So very helpful when you're trying to, um, get to, uh, show some sort of like inference and reasoning capabilities. You could also-- A good analogy on behavioral fine-tuning is it's very similar to LangChain functions, um, if, if you've used LangChain functions.

[16:02](https://www.youtube.com/watch?v=cXPYtkosXG4&t=962s) And parameter-efficient fine-tuning is like the standard fine-tuning where we are freezing some of the parameters, and we're only updating a very small amount of parameters using the techniques LoRA, QLoRA, and so on.

[16:17](https://www.youtube.com/watch?v=cXPYtkosXG4&t=977s) But coming to, you know, are these techniques going to really work? Sure, we have all of this available. It would only work depending on how good your data is, which is how-- It depends on how you're collecting your data, how you're tokenizing your data, how you're cleaning and normalizing your data.

[16:37](https://www.youtube.com/watch?v=cXPYtkosXG4&t=997s) Are you removing the noise and sort of sanitizing your models? Are you doing data deduplication as well to be able to, um, remove the duplicate entries? So there was another research that was published, which was basically like the memorization which happens in models is mainly because of data duplication.

[16:56](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1016s) Um, if, if we're removing the duplicate entries, that, that reduces the probability of a model to be able to memorize certain tasks, because again, it's seen those, uh, datasets over and over again in some form or the other.

[17:09](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1029s) So it's, it's naturally collecting-- creating sort of bias towards those things, and it's naturally outputting those very quickly. And the last one being data augmentation.

[17:21](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1041s) Now, let's say you've done all of this. Let's say you've picked the right model. Let's say you've done your data collection thing perfectly. You've got the best data out there.

[17:30](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1050s) What are still the things that you can think of while optimizing the performance of your model? So the first thing is do not try to compare with a GPT-4 or GPT-5.

[17:40](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1060s) It's not going to work comparatively, especially for more complex tasks. It's not a generalized model. Um, while it may be able to capture the nuances of your actual data, but it may not be able to capture the nuances of the new data that it hasn't seen or in newer domains that it hasn't seen before.

[18:01](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1081s) Um, so that's, that's one thing which I've seen a lot of companies are trying to-- are sort of in a dilemma with, uh, which is, "Oh, we've fine-tuned our model, but it's not working as good as a GPT-4."

[18:14](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1094s) The second one is basically using- In-context learning with dynamic examples. And one of the big reasons for that is, um, the, the big problem that we see with the drift in the model-- with the data drift in the models.

[18:30](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1110s) So using in-context learning with dynamic example loading allows you to be able to deal with that par-particular problem while also making sure that you are able to do, uh, cost management as well.

[18:46](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1126s) Um, the third thing that also one needs to think of is breaking down this task into smaller tasks. So for example, like if we are working with any sort of language, then instead of trying to train the model for like the entire language, can we break it down into like very specific tasks?

[19:02](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1142s) Um, so that's, that's another thing which people need to think of. Um, the final thing I would say is, uh, implementing some sort of gradient checkpointing. So what gradient checkpointing essentially does, it, um, it reduces the memory usage.

[19:19](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1159s) Um, what it, what it essentially does is it retrains the model, uh, and recomputes the weights during the backward pass. While it may look like, you know, it's, it's not the, it's not the smartest choice to make, um, but, you know, while the computation is higher, which is, yes, the, the weights will need to be

[19:44](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1184s) recomputed, um, but the downsides are easily outweighed by the memory consumption. So the memory consumption is very, very less if we are implementing some sort of gradient checkpointing. So another cost effect, uh, cost management thing.

[20:00](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1200s) Um, now few more considerations and limitations, which is let's talk about the hyperparameters now. Choosing a batch size. Ideally, we go with a batch size of thirty-two or sixty-four.

[20:12](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1212s) Uh, choosing the number of training epochs. Again, one of the questions I often get is what's the right number of epochs that we should be training with? Um, if you're, if you're doing a simple test, which is if you're running something in a Google Colab, uh, for fun thing, maybe having epoch one is nice.

[20:33](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1233s) Uh, but if you're, if you are working with a good model and if you're trying to optimize for like a particular domain, then choosing to go with hundred epochs as like the starting point is, is probably like the ideal choice.

[20:47](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1247s) Choosing an optimizer. There are different optimizers that are out there. Uh, Adam optimizer is the standard choice because it's general purpose, and it works really well with different domains as well.

[20:59](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1259s) Um, implementing some sort of regularization, early stopping. Again, uh, one of the things is basically like in, in terms of if, if you're looking at the models that have been trained till now, they're, they're not a lot of-- there's not a lot of implementation on optimizing those performances.

[21:17](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1277s) While there are bigger models that we are seeing every single day with more and more parameters, it-- they're not essentially squeezing all the performance out of those models. So one of the easy ways to be able to do that is using some sort of early stopping, which is making sure that you're only working with the data that

[21:37](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1297s) is most efficient. If the model performance is declining, then you need to reconsider your batch and look into that batch, consider your embeddings.

[21:48](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1308s) Now, let's say if you fine-train, uh, fine-tune the model, the next part, which is the hardest part of the process, is, um, you know, how do we evaluate our models?

[21:57](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1317s) There are so many, um, benchmarks out there. There are so many libraries out there. Um, so there's Aviary by Ray, there's, um, libraries j-- by NVIDIA. Um, but what you're essentially looking for mostly is the loss accuracy and perplexity, but that doesn't really paint the full picture.

[22:17](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1337s) So while I say, you know, it is the hardest part, which is there needs to be some sort of adaptation for every single business and every single use case, which is we need to be looking at evaluation from four different perspectives or four different components.

[22:33](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1353s) The first is doing some sort of metric-based evaluation, which is something like BLEU score, ROUGE score that we were considering before. Doing some sort of tool-based evaluation. So I think Weights & Biases does have a library for doing that particularly, which is their auto evaluate-- the debugger one.

[22:50](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1370s) And then there's another one, Auto Evaluator. Um, so that is able to catch the compilation errors very quickly. The, the third one is using some sort of model-based evaluation, which is using a smaller model to be able, able to evaluate the other model.

[23:05](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1385s) So while this is something which is, um, I've, I've not seen a lot of performance with this one because, again, it's hard to do, but it has a lot of potential, which is it does standardize the process eventually, and it automates the process.

[23:22](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1402s) And the final one is basically human-in-the-loop, which is something I feel like, you know, this is something that we are-- everybody is doing, um, but not the most efficient.

[23:31](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1411s) So let's, let's just ignore human-in-the-loop. Maybe let's let OpenAI talk about this. Um,

[23:40](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1420s) the final thing that I wanted to say on this one, uh, for this particular presentation is, um, while fine-tuning is great, yes, you-- but you also need to think about the entire pipeline, which is how you're thinking about the data collection, how you're thinking about the storage management, how you're choosing a base model.

[23:59](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1439s) So optimizing the performance of your model doesn't really depend on just one feature. While it may work perfectly for like a single one-off demo, but to be able to put a robust application that does sustain the test of time, and obviously, I, I'm, I'm not saying, you know, what would be an ideal time that you should be

[24:20](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1460s) testing on. Um, but in, in the case, the goal is to be able to get the optimal performance of the model and to be able to deal with all the data drift and the prompt drift and all of those things, while also making sure that we're catching a few things early and we're not exposing the enterprise to

[24:37](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1477s) like reputational risk, compliance risk, and all of those things. The entire thing has to be thought of. Um, so it is a big picture decision that I would say, um, that needs to be taken.

[24:48](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1488s) So that's, that's all my presentation for today. Um, I, I hope everybody learned something new. If there is, uh, something you would like to go with me in detail, then we can do that after the presentation.

[24:59](https://www.youtube.com/watch?v=cXPYtkosXG4&t=1499s) But thank you so much. [audience applauding] [upbeat music]
