Pre-training vs fine-tuning
Building an LLM happens in two distinct phases, and they teach the model fundamentally different things.
Pre-training is unsupervised. You feed the model billions of tokens of raw text and train it to predict the next token. This teaches the model language itself — grammar, facts, reasoning patterns, code syntax, even some world knowledge. But a pre-trained model has no concept of "instruction" or "answer." Ask it to summarize a paragraph and it might just continue writing more paragraphs.
Supervised fine-tuning (SFT) is the fix. You take the pre-trained model and train it on a dataset of (instruction, response) pairs. Now the model learns a new skill: given an instruction, produce a helpful response. This is the step that turns GPT into ChatGPT.
Think of pre-training as reading every book in the library. The model knows everything — but it doesn't know how to be helpful. SFT is like job training: you show the model thousands of examples of good behavior (clear answers, step-by-step reasoning, polite refusals) and it learns to mimic that pattern.
The remarkable thing is that SFT doesn't need much data. While pre-training requires billions of tokens, SFT typically uses only 10,000-100,000 high-quality instruction-response pairs. The knowledge is already there from pre-training — SFT just teaches the model how to access and present it.
Instruction following — the SFT dataset format
An SFT dataset is a collection of examples, each with three fields: instruction (what the user wants), input (optional context), and output (the ideal response). This is the format popularized by Stanford's Alpaca dataset.
LoRA — Low-Rank Adaptation
Fine-tuning all the parameters of a large model is expensive. A 7B-parameter model needs ~28GB of GPU memory just for the weights, plus optimizer states that triple the memory requirement. LoRA (Low-Rank Adaptation) solves this by freezing the base model and training only small adapter matrices.
Instead of updating a full weight matrix W (shape d x d), LoRA decomposes the update into two small matrices: A (shape d x r) and B (shape r x d), where r is the "rank" — typically 4, 8, or 16. The effective weight becomes W + A x B. Since r is tiny compared to d, this requires dramatically fewer trainable parameters.
When to fine-tune vs prompt
Not every task needs fine-tuning. With powerful base models, you can often get good results just by writing a better prompt. Here's when each approach makes sense.
| Factor | Use prompting | Use fine-tuning |
|---|---|---|
| Data available | Few examples (0-50) | Hundreds to thousands of examples |
| Task complexity | Can describe in natural language | Hard to describe, easier to show |
| Output format | Flexible / standard | Very specific format required |
| Latency | Long prompts are OK | Need fast responses (shorter prompts) |
| Cost per request | Higher (long prompts) | Lower (knowledge baked in) |
| Domain | General knowledge | Specialized domain (medical, legal) |
| Iteration speed | Minutes to try new approaches | Hours/days for each experiment |
Applying LoRA to an existing model
To fine-tune with LoRA, you freeze all base model parameters and replace selected linear layers with LoRA-wrapped versions. Typically you apply LoRA to the attention layers (Q, K, V projections) — these are the layers that most benefit from task-specific adaptation.
r controls the capacity of the adapter — rank 8 is a common default, but for complex tasks you might use 16 or 32.
SFT training loop
The training loop for SFT is nearly identical to pre-training — the only difference is the data. Instead of raw text, you feed instruction-response pairs. The model still trains with next-token prediction, but now it predicts the tokens of the response given the instruction.
Run it yourself
Run SFT with LoRA locally
Copy the code from the sections above into a single file and run it on your machine:
pip install torch python sft_training.py
Try changing the LoRA rank from 8 to 4 or 16 and observe how the trainable parameter count changes. With rank 4, you'll have ~6,144 trainable params (0.3%). With rank 16, you'll have ~24,576 (1.2%). The sweet spot depends on your task complexity.
Why this matters for LLMs
SFT is how every modern assistant is built
Every LLM assistant you've used — ChatGPT, Claude, Gemini, Llama Chat — started as a raw pre-trained model that could only complete text. SFT transformed each of them into instruction-following assistants. The technique you just learned is the exact same process, just scaled up.
LoRA has become the de facto standard for efficient fine-tuning. Hugging Face's PEFT library, QLoRA (which combines LoRA with 4-bit quantization), and frameworks like Axolotl all build on the same principle: freeze the base, train small adapters. Engineers who understand this pipeline are building custom AI products at companies across Africa, India, and Silicon Valley.
Challenge
Create 10 instruction-response pairs for a specific domain (e.g., customer support for an e-commerce store, medical Q&A, or code review). Format them using the instruction/input/output schema from this lesson. Think about what makes a "good" response — conciseness, accuracy, tone.
Modify the LoRA implementation to apply adapters to both the QKV projection AND the output projection in each attention block. How many trainable parameters does this give you with rank=8? How about with rank=4? Calculate the savings percentage for each configuration.
Quiz
You just built the SFT pipeline.
Now build the career to match.
Join the Best AI Program: The AI-Era Agentic Engineer — 5 months, fully online, with placement assistance. Week 1 is free.
Explore The AI-Era Agentic Engineer →