Mini Project: Fine-Tuned LLM with Instruction Tuning
End-to-end: Prepare 500 custom instruction examples, configure LoRA, train Llama 3.2 1B, evaluate with BLEU/ROUGE, run inference on new prompts.
Instruction Tuning
LoRA • Llama 3.2 1B
Model Evaluation
28 min read + 2 hour labAdvanced Project
🇬🇭
Meet Kwame
NLP Researcher in Accra, Ghana
Kwame is building a conversational AI for customer support in Twi (Ghanaian language) and English. He has collected 500 instruction-response pairs (mixed language support). His team wants a lightweight model that runs on edge devices (inference on phones/tablets). He chose Llama 3.2 1B (1.3B parameters) with LoRA fine-tuning—trainable on a single GPU, deployable on phones, and capable of domain-specific adaptation. This project walks through the entire pipeline: data preparation, training with torchtune, evaluation, and inference.
500
Instruction examples (domain-specific)
1.3B
Model size (lightweight)
~4h
Training time (1x GPU)
📋 Project Overview & Architecture
This mini project is split into 4 phases:
Phase 1: Data Preparation
Create or collect 500 instruction-response pairs. Format as JSON. Validate format, check for duplicates, explore length distributions. Output: instruction-tuning-500.json
Phase 2: Training Setup & Configuration
Download Llama 3.2 1B. Write a LoRA config (rank=8, alpha=16). Define optimizer, batch size, learning rate. Prepare a torchtune recipe or training script.
Phase 3: Training & Monitoring
Fine-tune on your 500 examples for 3-5 epochs. Monitor loss curves, validation metrics. Save checkpoints. Track training time and GPU memory usage.
Phase 4: Evaluation & Inference
Evaluate on held-out test set (100 examples). Compute BLEU, ROUGE-L. Run inference on new prompts. Compare before/after model outputs. Showcase improvements.
🔧 Phase 1: Data Preparation
Start by creating your instruction dataset. Here is Kwame's template for customer support scenarios:
Create instruction-tuning-500.json
Dataset validation output
Created 500 examples in instruction-tuning-500.json
Dataset stats:
Instruction length: 3-12 words
Output length: 15-85 words
Total examples: 500
Unique instructions: 498 (minimal duplication)
Distribution of instruction and output lengths (500 examples)
Instructions average 6-8 words, outputs average 35-45 words. This is typical for Q&A/customer support. Your model will learn to produce short questions and longer, detailed answers.
⚙️ Phase 2: Training Setup & LoRA Config
Configure LoRA parameters and training hyperparameters. For Llama 3.2 1B with 500 examples:
lora_config.py - LoRA Configuration
Why These LoRA Settings?
rank=8: Low rank for 1B model. Rank 16 or higher risks overfitting on 500 examples. rank=8 adds ~655K trainable params (0.05% of model). lora_alpha=16: Standard scaling (alpha/rank = 2). Controls LoRA contribution magnitude. target_modules=[q_proj, v_proj]: Query and value projections capture attention patterns. Including MLP would add more params but marginal quality gain for small dataset.
🚀 Phase 3: Training Loop
Use torchtune or Hugging Face Trainer. Here is the Trainer approach (simpler for small projects):
Training and validation loss curves across 3 epochs
Loss decreases smoothly from 3.12 to 1.73 (training) and 2.31 to 1.73 (validation). No overfitting—validation tracks training closely. Stable learning indicates good hyperparameter choices.
📊 Phase 4: Evaluation & Inference
Evaluate the fine-tuned model on test set using BLEU, ROUGE-L, and manual inspection:
evaluate.py - Compute Metrics
Evaluation metrics
BLEU-4 Score: 0.3542
ROUGE-L (F-measure): 0.5231
Sample predictions (test set):
Instruction: "How do I reset my password?"
Reference: "To reset your password, go to the login page..."
Generated: "Go to the login page, click 'Forgot Password', follow email steps."
Instruction: "What payment methods do you accept?"
Reference: "We accept Visa, Mastercard, PayPal, MTN Mobile Money..."
Generated: "We accept major credit cards (Visa, Mastercard), PayPal, and mobile money."
Before vs After fine-tuning: comparison of base model vs fine-tuned outputs
Base model (zero-shot) generates generic or irrelevant responses. Fine-tuned model produces domain-specific, helpful answers matching customer support context. BLEU/ROUGE improvements confirm better semantic alignment with reference answers.
🚀 Inference & Deployment
Deploy your fine-tuned model for inference. For edge devices (phones), use quantization:
inference.py - Inference Script
Inference output (live)
=== Fine-Tuned Llama 1B Inference ===
Q: How do I track my order?
A: Check your order confirmation email for a tracking link. Status updates will be sent via SMS and email. Processing typically takes 1-2 business days.
Q: What is your refund policy?
A: We offer a 30-day return policy. Items must be unused and in original packaging. Refunds process in 5-10 business days after we receive your return.
Q: Do you ship internationally?
A: Yes, we ship to 15 West African countries. Standard shipping (7-10 days) and Express (3-5 days) available. Free shipping on orders over GHC500.
Q: I forgot my password. What should I do?
A: Visit the login page and click "Forgot Password". You'll receive an email with reset instructions. Check your spam folder if you don't see it.
🏆 Mini Challenge: Extend the Dataset
Task: Collect 100 more instruction-response pairs in your domain (make them realistic—order status, billing issues, product questions). Retrain the model with 600 total examples (500 train, 100 test). Measure if BLEU and ROUGE improve.
Hint: More diverse data typically improves generalization. Check if the model generalizes to questions it hasn't seen before (e.g., variations of "How do I track my order?").
💼 Interview Questions
Instruction tuning fine-tunes a pretrained LLM on instruction-response pairs to teach it to follow domain-specific commands and tasks. It's important because pretrained models are general-purpose; instruction tuning adapts them to specific use cases (customer support, legal analysis, etc.). With just 500 examples, you can build a domain-expert model that outperforms zero-shot base models by 50-80% on quality metrics.
LoRA trains only 0.05% of model parameters (vs 100% for full fine-tuning). Benefits: (1) Memory: 28 GB → 8 GB for 7B models. (2) Speed: ~2x faster training. (3) Portability: Save only 10 MB adapter weights, not full 7 GB model. (4) Quality: 95%+ of full fine-tuning. For 500-5000 examples, LoRA is preferred because it prevents overfitting while maintaining performance.
BLEU (Bilingual Evaluation Understudy) measures n-gram overlap between generated and reference text (precision-focused). Ranges 0-1. ROUGE (Recall-Oriented Understudy for Gisting Evaluation) measures recall of n-grams and longest common subsequences. BLEU is better for translation; ROUGE for summarization. For question-answering, ROUGE-L (longest common subsequence) is more forgiving—exact phrasing varies but semantic meaning is key. Typical values: BLEU 0.25-0.4, ROUGE-L 0.45-0.65 for customer support.
Domain shift: Production queries differ from training (e.g., typos, slang, longer context). Data leakage: Test set is too similar to training (not held-out). Model brittleness: 500 examples may not cover edge cases (unusual questions, requests in Twi vs English). Solutions: (1) Collect production failures and retrain. (2) Create diverse test set. (3) Add data augmentation (paraphrase, backslation). (4) Use ensemble with keyword fallback. (5) Monitor in production with metrics.
(1) Quantization: int8 or int4 reduces model size from 2.6 GB to 1 GB (int8) or 650 MB (int4). Minimal accuracy loss (1-2%). (2) Pruning: Remove low-importance weights, further reducing size. (3) Knowledge distillation: Train a smaller model to mimic fine-tuned model, then quantize. (4) Inference optimization: Use llama.cpp or MLX (Apple Silicon), optimized for mobile. (5) On-device caching: Cache responses to common queries. (6) Hybrid approach: Simple classifier selects predefined templates for 80% of queries (customer support), use LLM for 20% (complex). Final deployment: ~200-500 MB on-device, <1 sec inference on modern phones.
✓ Quick Quiz
Question 1 / 5
You have 500 labeled instruction-response pairs. How should you split them for training and evaluation?
Question 2 / 5
What LoRA parameters are appropriate for a 1B model with 500 examples?
Question 3 / 5
Why should you mask loss on instruction tokens (label them -100)?
Question 4 / 5
Your fine-tuned model achieves BLEU 0.35 on the test set. Is this good?
Question 5 / 5
To deploy your fine-tuned Llama 1B to phones (2 GB limit), what is your strategy?
Saves 10-50 MB; merge with base model for inference
Loss masking
labels[:, :instruction_len] = -100
Prevents loss on input; only output tokens contribute
⚠️ Common Mistakes
Not masking instruction tokens during loss computation
If you compute loss on instruction tokens, the model learns to predict instructions (copying input), not generate novel responses. Always set labels[:, :instruction_len] = -100 to mask instruction tokens.
Using too high learning rate for LoRA
LoRA is learning adapters on top of frozen weights. Learning rate 1e-3 (standard) is safe. If loss explodes, reduce to 5e-4 or 3e-4. High LR can corrupt adapter weights and cause poor fine-tuning.
Not splitting train/test before data inspection
If you inspect all 500 examples, then split, test set leakage occurs—model has seen examples it evaluates on. Always split first, then explore only training data.
Relying on BLEU alone for evaluation
BLEU is n-gram overlap and can be misleading (e.g., paraphrasing gets low BLEU despite high semantic quality). Use BLEU + ROUGE-L + manual evaluation.
Ignoring class imbalance in instruction types
If your 500 examples are 400 "How do I..." and 100 "Tell me about...", the model may overfit to "How" questions. Ensure diverse instruction types and evaluate per category.
✦ Key Takeaways
Instruction tuning adapts pretrained LLMs to domain-specific tasks with 500-5000 examples. Results in 50-80% quality improvement over zero-shot.
LoRA fine-tunes only 0.05% of parameters, reducing memory and training time by 60-70%. Quality is 95%+ of full fine-tuning for small datasets.
Always mask loss on instruction tokens—force the model to generate novel outputs, not copy inputs. Set labels to -100 for instruction positions.
Evaluate with multiple metrics: BLEU (n-gram overlap), ROUGE-L (recall-focused), and manual inspection. No single metric captures quality.
Deploy to mobile with int8 quantization (2.6 GB → 1 GB) for on-device inference. Balance accuracy and size for edge devices.
Kwame's fine-tuned Llama 1B achieves BLEU 0.35 + ROUGE-L 0.52 on customer support Q&A, generalization to unseen questions, and 4-hour training on 1x GPU.