Why "seems to work" is not a metric
You build an agent that calls a ticker lookup tool. You test it manually on RELIANCE.NS, it returns price and news—looks good. Ship it. Then a user asks for SBI.NS, and your agent crashes because the tool endpoint timeout logic wasn't tested. Another user asks for an obscure ticker; the agent hallucinates a stock price instead of saying "I don't know."
Arjun ships a ticker agent for SouthBank India's retail trading platform. On his machine, it works fine. But in production, 15% of user requests fail or return wrong data. Why? Because he never built a systematic test set. He never measured latency or token costs. He never asked: "What should my agent return for ticker X, and did it?"
He rewrites the agent with a 20-ticker eval set, deterministic scoring (price match, news count, sentiment accuracy), and LLM-as-judge grading. Now he catches regressions before they ship. Every time he tweaks the system prompt, he runs the eval suite. Score goes from 65% to 78% in two weeks.
Evaluation is how you separate "my agent works on my test" from "my agent works reliably." It's the backbone of any AI system that touches production.
Four types of agent evaluation
Not all tests are equal. Build a stack:
| Type | Method | When to use | Cost |
|---|---|---|---|
| Unit tests | Test each tool independently: does `get_price(RELIANCE)` return a dict with price, symbol, timestamp? | Every tool change. Catches silent failures. | Zero (no LLM calls) |
| End-to-end | Feed the agent a user query, check final output against ground truth | After system prompt changes. Measures real user experience. | 1 LLM call per test case |
| Deterministic scoring | Exact match, substring match, numeric tolerance, schema validation | When you have ground truth (price must be between ₹2500–2600) | Zero (rule-based) |
| LLM-as-judge | Use a stronger model (Gemini, GPT-4) to grade your agent's output | When ground truth is fuzzy ("Does this summary capture the key point?") | 1 judge call per test case |
A robust eval harness uses all four. Start with unit tests (free), then add end-to-end tests (cheap), then layer in deterministic scoring and LLM judges for subjective cases.
Step 1: Build a realistic test set
Your test set is a list of (input, expected_output) pairs. For a ticker agent, that might be 20 tickers with known prices, news counts, and sentiment. For a customer support agent, it might be 30 customer emails with known categories and priorities.
Key principles for test set design:
- Realistic scope. 20–50 test cases. Small enough to run in seconds, large enough to catch patterns.
- Known outputs. For each test case, you must know what a correct answer looks like.
- Boundary conditions. Include edge cases: empty results, timeouts, ambiguous tickers (is "TCS" unambiguous?), malformed inputs.
- Representative diversity. Don't test only the stocks you're familiar with. Include rare tickers, cross-sectors, unstable prices.
Step 2: Deterministic scoring (rule-based, zero LLM cost)
When you have ground truth, score with rules. No LLM needed. Fast and reproducible.
The scoring rubric is tuned to your domain. For financial data, price tolerance matters. For news summaries, you might score on keyword presence or sentiment accuracy. The key: each rubric item is deterministic.
Step 3: LLM-as-judge for subjective quality
When ground truth is fuzzy—"Is this summary good?"—ask a stronger LLM to grade. Use a model like Gemini 2.0 Flash (faster, cheaper) to judge your Ollama agent's outputs.
LLM judges are expensive (1 judge call per test case), so use them sparingly. Combine with deterministic scoring: run deterministic tests first (free), then judge only the borderline cases.
Step 4: Measure cost and latency
Wrap every agent call with time and token accounting. Over time, you'll see patterns: "This system prompt causes 2x token usage, no quality gain."
Track these in a spreadsheet or database. Over 10 eval runs, you'll see cost drift. If your average tokens per call went from 150 to 300, your system prompt change doubled token usage. This is actionable feedback.
Step 5: Regression testing—rerun eval suite on every change
Every time you change the system prompt or add a tool, rerun the full eval suite. Track results in a log. If the score drops, you caught a regression before it ships.
Keep a CSV log:
| Timestamp | Change | Avg Score | Avg Latency (ms) | Avg Tokens |
|---|---|---|---|---|
| 2026-04-01 10:00 | Baseline | 0.74 | 245 | 180 |
| 2026-04-01 11:30 | Add sentiment check | 0.81 | 268 | 195 |
| 2026-04-01 14:00 | Verbose system prompt | 0.68 | 450 | 380 |
| 2026-04-01 15:00 | Revert to add sentiment | 0.81 | 268 | 195 |
Worked example: Full eval harness for ticker briefing agent
Let's build a complete eval pipeline for the Lesson 05 ticker agent. We'll define 10 tickers, score deterministically, run the agent, and measure quality.
Step 6: A/B testing prompt variants
Build two system prompts. Run them against the same test set. Which one scores higher?
When to trust LLM-as-judge vs when to need humans
| Scenario | Trust LLM Judge? | Also need humans? |
|---|---|---|
| Pricing accuracy ("Is price within ±2%?") |
✗ No | Deterministic rule only. LLM might hallucinate. |
| News relevance ("Does news relate to the company?") |
✓ Yes | LLM is good at this. Sample human review monthly. |
| Tone/sentiment classification ("Is the briefing friendly?") |
~ Moderate | Use 2-judge approach: Claude + GPT. Agree = trust. |
| Format correctness ("Is JSON valid? All fields present?") |
✗ No | Use Pydantic schema validation. Rule-based only. |
| Safety/toxicity ("Does briefing contain slurs?") |
✓ Yes, but sample | Judge 20% of test set weekly. Audit failures monthly. |
Rule: Use LLM judges for subjective quality (relevance, clarity, completeness). Use deterministic scoring for objective facts (format, price range, field presence). When stakes are high (financial, healthcare), always include human audit loop.
Task: Define a test set for 5 stocks (RELIANCE.NS, HDFCBANK.NS, INFY.NS, TCS.NS, SBIN.NS) with price ranges and minimum news counts. Write a scoring function that gives each test case a 0–1 score based on price accuracy and news count. Run your ticker agent from Lesson 05 on all 5 tickers and report average score and latency.
Hint 1: Use the test set structure from this lesson. Define price_range and min_news per ticker.
Hint 2: Scoring: 0.5 points for price in range, 0.5 for news count >= min_news.
Hint 3: Wrap agent calls with time.time() to measure latency.
Interview Q&A: Agent Evaluation
Test your knowledge
You've built eval harnesses.
Now ship agents that scale.
Join the Best AI Program: The AI-Era Agentic Engineer — 5 months, fully online, with placement assistance. Week 1 is free.
Explore The AI-Era Agentic Engineer →