What this mini-build is

Lessons 05 and 06 gave you the theory: a prompt is a spec, and guardrails are the contract. This lesson is the kitchen. You will sit next to Priya — the SouthBank India PM you met in Lesson 00 — and walk through 90 minutes of her Tuesday afternoon. At 10am she had a failing prompt and a launch date. By 11:30am she has a prompt that clears the gate. Nothing else about her world changed. Only the prompt.

You will not write code. You will make six prompt decisions: one per part (role, context, task, format, refusals) plus the few-shot examples that glue them together. For each decision you will see the before line, the after line, and the exact golden-set failure that forced the change. That is the job. The model did not get better between v1 and v3. Priya did.

The one-sentence frame:
A production system prompt is not written — it is refactored, against a golden set, one failure family at a time, until the pass rate clears the gate.

Priya's Tuesday — the launch is Friday

🏦
Real-world scenario
SouthBank India · Mumbai · support bot must hit 85% by Friday

PM Priya Menon leads the retail-banking AI squad at SouthBank India. Ten weeks ago she scoped a chat support bot: "answer the top-40 account questions so we deflect 30% of Tier-1 tickets." Today is Tuesday of week 10. Friday is the canary launch. Her golden set has 100 real user questions from the 2026 April tickets, hand-labelled with a reference answer by two CX leads and a compliance officer.

Her v1 system prompt — the one she wrote in 45 minutes three weeks ago between standups — scored 61% on the golden set. The launch gate is 85%. Her VP of Product has blocked the 3pm "kill the launch" meeting on her calendar. She has 90 minutes before that meeting. This lesson is those 90 minutes.

v1 — the prompt that failed the gate

Here is Priya's v1, colour-coded by the five-part anatomy. Read it first. Notice where you would flinch if a junior PM on your team handed it to you.

v1 · Broken 61% pass · fails on refusal, format, and ambiguity
What the judges flagged:
12 answers invented account features that do not exist ("you can pause your debit card from the settings menu" — SouthBank has no such toggle). 9 answers tried to assess loan eligibility with confident rupee figures. 6 answers used the casual phrase "no worries!" which the brand team rejected. 4 answers were five paragraphs long. 8 answers had no source anchor. The other 22 failures were split across those categories. None of this is a model problem. It is a prompt problem.

The refactor loop — five moves, one golden set

Priya does not rewrite the prompt from scratch. That is a beginner move. She runs a tight loop: look at the failure cluster, change exactly one part of the prompt, re-score, move on. Five passes in 90 minutes.

1. Write v1 (45 min once) 2. Run eval 100 Qs · 61% pass 3. Read failures cluster, not count 4. Refine 1 part not the whole thing 5. Re-eval loop to ≥85%
The discipline: change exactly ONE part of the prompt per iteration. If you rewrite three parts at once and the score jumps, you have no idea which change did it — which means you cannot reason about the next one.

The 90-minute build — one part at a time

Here is Priya's screen for each of the five passes. For every part of the prompt you get the before line, the after line, and a one-sentence "why this works." Read this top to bottom — the shape of the prompt emerges as you go.

1
Role — narrow the job until the refusals write themselves
Pass 1 · ~12 min · fixes 6 "out-of-scope" failures

A fuzzy role ("helpful assistant") gives the model permission to answer anything — which is exactly why it cheerfully invents a pause-card feature. Priya narrows the role to the smallest verb set that still covers the top-40 ticket categories, and names what the bot is not in the same breath.

v1 · before
You are a helpful SouthBank customer assistant.
v3 · after
You are SouthBank India's Tier-1 support assistant. You explain existing account features, transactions, UPI status, and card states. You are NOT a loans officer, investment adviser, grievance officer, or a channel to change any account setting.
Why it works: the second clause ("NOT a loans officer…") is what actually makes the refusal rules further down enforceable. A narrow role is the cheapest guardrail you own.
2
Context — give the model what only you know
Pass 2 · ~15 min · fixes 12 "invented feature" failures

The v1 context told the model SouthBank is a bank. The model already knew that. What it did not know is which features exist in the SouthBank app, which are on the roadmap, and which are permanently off the table. Context is where the PM pays the "what does our product actually do" tax so the model stops making things up.

v1 · before
SouthBank is a bank. Users may ask about their account, cards, loans, UPI, or anything else.
v3 · after
Grounded facts (only cite these; do not add any other): (1) SouthBank offers savings, current, NRE, NRO accounts. (2) Debit-card controls available in-app: block card, reset PIN, set international-use toggle. Pause-card is NOT available. (3) UPI issues route to the PhonePe/GPay app first; SouthBank can only confirm if the debit to the account succeeded. (4) Loan applications are handled only on the loans portal; this bot never quotes rates or eligibility.
Why it works: every factual claim the model can make is now on an allow-list. The 12 "invented feature" failures in v1 all came from the model pattern-matching on generic banking-app behaviour. Explicit, product-specific context kills that.
3
Task — turn "answer the question" into a five-step procedure
Pass 3 · ~18 min · fixes 8 "no citation" + 4 "too long" failures

"Answer the user's question" is not a task — it is a wish. A task is a procedure. Priya replaces the v1 one-liner with an explicit five-step routine that tells the model exactly how to decide between answering, citing, narrowing, and escalating.

v1 · before
Answer the user's question as best you can. If you don't know, try to help anyway.
v3 · after
For every user message, in order: (a) classify the intent into one of {account_info, card_state, upi_status, loan_request, grievance, off_topic}; (b) if intent is loan_request or grievance, use the refusal rule below and stop; (c) otherwise, answer ONLY using facts from the grounded-facts block or the retrieved account record; (d) cite the fact ID you used; (e) if the retrieved record is missing, respond with the "need more info" template and ask for the single most useful detail.
Why it works: citation becomes a required step, not a hope. "Help anyway" — the phrase that generated 8 uncited answers — is replaced by a deterministic fall-through to a template. The model has nowhere to improvise.
4
Format — pin the shape the CX team can actually render
Pass 4 · ~10 min · fixes 4 "too long" + 6 "wrong tone" failures

"Short and clear" in v1 meant whatever the model felt that day. Priya pins the output shape to exactly what the chat bubble and the analytics pipe can consume — a fixed length budget, a tone label the brand team already signed off on, and a machine-readable citation slot.

v1 · before
Keep answers short and clear.
v3 · after
Answer in 2-4 sentences, maximum 60 words. Tone: calm, factual, second-person ("you"). Never use "no worries", "totally", emojis, or exclamation marks. End every answer with a citation tag in square brackets, e.g. [fact:2] or [record:txn_12345]. If you cannot cite, do not answer — use the "need more info" template.
Why it works: length, tone, lexicon, and citations are now all machine-checkable. The eval harness can now assert "ends with [fact:N]" with a regex — no LLM judge needed for that dimension.
5
Refusals & few-shot — rehearse the hard cases literally
Pass 5 · ~22 min · fixes 9 "loan advice" + remaining drift

The refusal rule and the two or three few-shot examples are the most important 300 words in the prompt. Priya writes the exact refusal string (not a paraphrase), then shows the model what success and failure look like with two literal dialogues.

v1 · before
Don't say anything inappropriate.
v3 · after
If the user's message falls under {loan_request, grievance, investment_advice, legal_question, pause_card_request, rate_quote}, reply EXACTLY: "I can only help with existing account, card, and UPI questions. For this, please tap 'Talk to an agent' below and a SouthBank executive will call you back within 30 minutes." Do not add anything else. Do not apologise twice. Do not explain why.
Why it works: the refusal is now a string, not a concept. The eval harness can assert response == REFUSAL_STRING for loan questions and get a hard pass/fail instead of an LLM-judge opinion.

Priya also appends two few-shot dialogues — one "good answer" and one "correct refusal" — inside the prompt. These are not decorative. They anchor the model's style more reliably than 300 extra words of instruction.

v1 · before
(no examples)
v3 · after
Example 1 — USER: "Why was ₹500 debited yesterday?" ASSISTANT: "A ₹500 debit on 22 Apr was for your SIP to ClearWealth Mutual. It's a monthly standing instruction you set up on 14 Feb. [record:txn_99421]" ·· Example 2 — USER: "Am I eligible for a ₹5 lakh loan?" ASSISTANT: "I can only help with existing account, card, and UPI questions. For this, please tap 'Talk to an agent' below and a SouthBank executive will call you back within 30 minutes."
Why it works: few-shot examples teach the model tone and shape that are hard to write as rules. Two examples beat ten adjectives. Pick them from real failing golden-set rows so you fix a failure family and anchor the pattern in the same move.

v3 — the prompt that clears the gate

Stitched together, here is the shape of v3. Same five parts, same colour code, very different density — and a sixth tag for the few-shot examples. Notice that the prompt is now roughly 3x longer than v1, and that the length is entirely in context, task, and refusal. Role is still two sentences.

v3 · Production 89% pass · clears the 85% gate with 4 points of headroom
What changed structurally:
role shrank, context grew, task became procedural, format became machine-checkable, refusal became a string, and two few-shot examples replaced the v1 "don't say anything inappropriate." Every pass moved the score; no pass rewrote the whole prompt.

The score after each pass

Here is the golden-set pass rate after each of Priya's three checkpoints. v1 is the Tuesday 10am starting point. v2 is the halfway save — after the role, context, and task passes. v3 is what she submits for canary at 11:30am.

Golden-set pass rate — 100 hand-labelled tickets v1 (10:00am) 61% v2 (10:45am) 74% v3 (11:30am) 89% ↑ Launch gate 85% Each bar = % of 100 Q/A pairs where the answer met the rubric (citation, length, tone, refusal, accuracy).
What Priya ships: v3 to canary at 1% traffic, with the same eval wired to re-run on every prompt PR. She does not declare victory on a 89% number — she declares victory when the gate holds for two consecutive days of canary traffic and the human-reviewed refusal precision stays above 95%.

v1 vs v3 — six dimensions

A clean lens on what changed. Use this as your review checklist when a colleague hands you a draft system prompt.

Dimensionv1 · brokenv3 · production
Length~90 words. Entirely role and wishes.~260 words. Length sits in context + task + refusal, not role.
Refusals"Don't say anything inappropriate." One line, no trigger list, no string.Six named intent classes trigger ONE literal refusal string. No paraphrase allowed.
Format"Short and clear." Untestable.2-4 sentences, ≤60 words, banned lexicon list, mandatory citation tag format.
ExamplesNone. The model is guessing at tone.Two literal dialogues: one good-answer, one correct-refusal. Pulled from real golden-set rows.
CitationsNot mentioned. 8 answers in v1 had none.Every fact must end with [fact:N] or [record:ID]. Regex-checkable.
EscalationNot mentioned. Model invented its own advice for loans.Explicit fall-through: "tap Talk to an agent, callback within 30 min." Handoff has an SLA.

The production prompt (copy this shape)

The full v3 prompt as a single block. This is the only code in this lesson. Read it, then steal the skeleton — the five headers and the examples at the end — for your own v1.

v3_system_prompt.txt prompt · production

The AI connection

What real teams do with this

Frontier AI teams version prompts like code — every change is a PR, gated by the golden-set pass rate and a refusal-precision threshold. The prompt file lives next to the eval file. The CI bot blocks merges that drop any failure-family score below its previous best. The PM's job is not to author perfect prose; it is to cluster failures, pick the cheapest intervention per cluster, and defend the gate. golden-set prompt-versioning refusal-precision few-shot-rehearsal

Your 20-minute mini challenge

Run Priya's refactor on your own support bot

1. Write the v1 in 10 minutes, five parts only

Pick any product you ship (or would ship). Write a v1 system prompt with exactly five labelled blocks: ROLE, CONTEXT, TASK, FORMAT, REFUSAL. Do not polish it. It should fit on one screen.

2. Hand-write a 10-row golden set

Ten real questions a user might ask, with a one-sentence reference answer for each. Include two off-scope questions and one question that needs a refusal. These ten rows are your launch gate.

3. Refactor ONE part, mark the expected failure-family fix

Read your v1 against your 10 rows. Pick the single cluster of failures (usually refusals or citations) and refactor only that one part. Write down the exact failure family you expect to fix. If you cannot name it, you are not ready to ship.

Interview Q&A

EasyWalk me through how you would go from a v1 system prompt to v2.
+
Run the v1 against a small hand-labelled golden set (50-100 rows is fine for a first pass). Read the failures — not the pass rate, the failures — and cluster them into 3-5 families like "no citation", "invented feature", "wrong tone", "missed refusal". Pick the one family with the highest count and the cheapest fix, edit exactly that part of the prompt (role, context, task, format, or refusal), and re-run the eval. That is v2. Repeat until you clear the gate. The discipline is changing one part per iteration so you can reason about cause and effect.
EasyWhy are few-shot examples more effective than more adjectives in the instructions?
+
Adjectives like "calm" or "concise" are subjective — the model has to guess where your dial sits. A literal example dialogue shows the exact tone, length, sentence structure, and citation style you want, and the model mimics shape faster than it parses instructions. Two well-chosen examples usually beat a paragraph of style guidance, and they double as regression tests — if the model stops matching the example style after a prompt change, you know the change hurt something.
MediumYour eval shows 87% pass rate — above the 85% gate. Do you ship?
+
Not on the pass rate alone. The 85% is necessary, not sufficient. Before shipping I would check three more things: (1) refusal precision — of the rows that required a refusal, did the model refuse with the exact string every time, or did it paraphrase? Paraphrase rate > 2% is a ship-blocker. (2) failure family distribution — 13% failing is fine if they are spread across families, but if 10 of the 13 are "leaked PII" I hold the launch. (3) variance — I run the eval three times; a 87% ± 5% prompt is not the same as a 87% ± 1% prompt. Only after all three do I move to canary at 1%.
MediumEngineering wants to fine-tune instead of prompt-refactoring. What's your counter?
+
Fine-tuning is a legitimate escalation — but only after prompt-refactoring has plateaued on the golden set and you understand why. I ask: have we closed refusal precision and format checks with the prompt alone? If yes, the remaining gap is usually reasoning or domain-specific phrasing, and fine-tuning helps. If no, fine-tuning will bake the same ambiguity into weights and make regression harder to reason about. Prompting is also 10x cheaper to iterate on — a prompt change is a PR, a fine-tune is a week plus a bill. Default to prompt until it stops moving the number.
HardYou refactor five versions and cannot break 83%. The launch is on Friday. What do you do?
+
First, stop iterating. Five versions without a jump means the remaining failures are not prompt-addressable — they are model, data, or scope failures dressed up as prompt failures. I do three diagnostics in parallel. (1) Is it a model ceiling? Run the same v5 prompt on a stronger model (e.g. move from 4o-mini to 4o). If the score jumps above 85%, the original prompt was fine — the model was the bottleneck and we have a cost vs quality decision to surface to eng leadership. (2) Is it a data ceiling? Read the 17 failing rows. If >5 of them require information that is simply not in the retrieved record, this is a retrieval / schema problem, not a prompt problem — fix it at the orchestration layer, not by writing more instructions. (3) Is the scope wrong? If three failures are "user asked a question we did not design for", the honest answer is to narrow the shipped scope to the 60 questions we can nail and refuse the other 40 cleanly — ship 100% on a smaller surface rather than 83% on the planned one. I go to the VP with those three lenses and a recommendation, not with "we need another week."

Quiz — 5 questions

Question 1 of 5
Priya's v1 scored 61% on the golden set. What should her FIRST iteration change?
Question 2 of 5
Which v3 change most directly fixed the 12 "invented feature" failures (e.g. "pause your debit card")?
Question 3 of 5
Why does v3 require the refusal reply to be an EXACT string rather than "something like" it?
Question 4 of 5
You ship v3 at 89% pass rate. On day 2 of canary, the rate drops to 78% with no prompt change. Most likely cause?
Question 5 of 5
Which is the strongest argument for adding two few-shot examples to v3 rather than two more paragraphs of style guidance?
0
/5
You can now ship a prompt that holds up. Ship the PR.

Common v1 mistakes — critique the draft

"Be helpful" as the entire role
"You are a helpful assistant" is an invitation to answer anything. A production role always has a NOT clause — the jobs the bot refuses to do. If you cannot write the NOT clause, you have not scoped the feature.
Rewriting everything between iterations
When the score jumps 14 points and you changed four sections at once, you learned nothing. You now have a v2 you cannot debug. Change one part per pass or you are gambling, not engineering.
Refusal as a concept, not a string
"Politely decline out-of-scope questions" produces a different polite decline every time, half of them inventing a refund policy while they apologise. Pin the literal string. Make the eval a regex.
No few-shot examples on a tone-sensitive bot
Tone is the first thing to drift and the hardest to regulate with instructions. Two real example dialogues pulled from past failing rows — one good answer, one correct refusal — will outperform any amount of prose about "calm and factual".

Key takeaways

  • A production system prompt is refactored, not written — five passes against a golden set, one part changed per pass.
  • Role is short. Context, task, and refusal are where the words live — that is where the failure-family fixes go.
  • Refusals are literal strings, not concepts. If the eval cannot assert response == REFUSAL_STRING, the guardrail is a wish.
  • Two few-shot examples beat two paragraphs of style guidance, and they double as regression tests on every prompt change.
  • Clearing the gate once is not shipping — watch refusal precision, failure-family distribution, and variance before you go to canary.
Africa & India's Best AI Academy
🤖 AI-first 🎓 Placement Assistance

You can now ship a system prompt that holds up.
Next — design the UX that wraps it.

Join the Best AI Program: The AI-Era Agentic Engineer — 5 months, fully online, with placement assistance. Week 1 is free.

Explore The AI-Era Agentic Engineer →