The problem: silent failures

An agent fails. The user sees "Error." You get a log line: "Agent crashed." That's it. You don't know which LLM call failed. You don't know what tokens cost. You don't know if it was a hallucination or a real error. You certainly can't replay the failure and debug it.

In production—especially in Kenya and India on 3G networks where latency spikes matter—agents fail in subtle ways. A tool call hangs. An LLM response is truncated mid-token. The trace disappears. If you haven't instrumented your agent with structured logging and tracing, you're flying blind.

Agent starts
→
LLM call
→
Tool call
→
Error ✗

What you see: "Error"
←
What you need: full trace

This lesson teaches you to see everything: every LLM call, every token, every tool invocation, every error with full context.

Structured logging with Python's logging module

The foundation is Python's built-in logging module with a JSON formatter. Instead of free-text logs like "Agent thought for 2.3 seconds," log structured JSON: one log line per action, parseable by machines.

One log = one action. Log when:

  • An LLM API call starts (model, tokens)
  • An LLM API call finishes (latency, cost)
  • A tool call starts (name, args)
  • A tool call finishes (result, duration)
  • An error occurs (stack, context)
structured_logging.py Logging setup with JSON formatter
Output (one line)
{"timestamp": "2026-04-16T14:32:10.123456", "level": "INFO", "logger": "agent", "message": "LLM call completed", "extra": {"model": "gpt-4", "prompt_tokens": 150, "completion_tokens": 75, "latency_ms": 3200, "cost_usd": 0.0045}}
Why JSON?
JSON logs are machine-parseable. Pipe them to ELK, CloudWatch, Datadog, or a pandas notebook. Query them instantly. No regex parsing needed.

Logging every LLM and tool call

Wrap your LLM and tool calls with logging. Capture what went in, what came out, how long it took, and how much it cost.

instrumented_agent.py Agent with full logging instrumentation
Cost tracking note:
Every LLM call has a price. Claude 3.5 Sonnet: ~$0.003 per 1K input tokens, ~$0.015 per 1K output. Log it. Sum by model, day, agent. You'll spot runaway costs fast.

Distributed tracing: one trace per user turn

Logs are events. Traces are the full call graph. One trace correlates all logs (LLM, tool, errors) into a single narrative. Assign a trace ID at the start of a user turn, pass it through every call, and you can replay the entire failure.

Trace anatomy:

  • Trace ID: UUID for the user turn (e.g., "turn-2026-04-16-a7f3d8")
  • Parent span: The agent's decision-making (e.g., "agent_turn")
  • Child spans: LLM call, tool 1, tool 2, error (if any)
  • Attributes: model, tokens, latency, cost, result

You can build traces with OpenTelemetry (the open standard) or a homegrown Tracer class. Here's a simple tracer:

simple_tracer.py Minimal tracer class

Real scenario: replay a failed trace

🔍
Real scenario
M-Pesa Kenya bot hallucination

A customer on 3G in Nairobi asks: "Transfer ₹500 to my friend's account." The bot should reject it (wrong currency). Instead, it says "Done." A trace shows:

  • LLM turn 1: "I need the account number" (correct)
  • Customer: "Send to 254-XXXXX" (provides number)
  • LLM turn 2: Latency 8,500ms (high—network lag?)
  • LLM output truncated at "I'll transfer KSh 500 to ..." (hung mid-response?)
  • Agent assumes it succeeded and logs "Transfer complete" (hallucination!)

Without traces, you'd never know the response was truncated. With traces + logs, you see the 8.5s latency spike and the incomplete response. You can reproduce it locally, fix the network timeout, and deploy.

PII redaction: scrub before logging

Never log phone numbers, emails, or account IDs directly. Redact them before sending logs to external systems.

redact_pii.py Remove PII before logging
Compliance:
Redacting PII keeps you GDPR-compliant. If logs leak to CloudWatch or a third-party analytics service, no secrets are exposed.

Free tools: Langfuse, Weights & Biases, ELK stack

Logging and tracing are useless without a backend to store and query them. Here are three free options:

Tool Cost Best for
Langfuse (open source) Free / self-hosted LLM tracing, cost tracking, replay UI
Weights & Biases Free tier + paid Experiment tracking, dashboards, real-time monitoring
ELK Stack (Elasticsearch + Logstash + Kibana) Free / self-hosted Raw log storage, ad-hoc queries, full-text search
CloudWatch (AWS) Pay-as-you-go (~$0.50/GB) If you're already on AWS, built-in integrations

For startups in Kenya and India, self-hosted Langfuse or a local ELK stack saves costs and keeps data private.

Token and cost dashboards with pandas

Write a simple Python notebook to group logs by model, day, and agent. Spot trends. Catch runaway costs.

cost_analysis.py Summarize daily costs by model

Why observability matters in Kenya and India

On 3G networks with variable latency, LLM calls can hang. Traces reveal:

  • Which calls timeout first (hint: it's often the longest prompts)
  • Latency percentiles (p50, p99) by time of day
  • Partial responses (e.g., LLM response cut short mid-sentence)
  • Tool call failures due to network interrupts

With traces, you can set intelligent timeouts: if an LLM call exceeds p99 latency + 2σ, fallback to a cached response or escalate to a human.

How AI systems use observability

Large language models are black boxes. They hallucinate. They fail silently. Observability is your window into the model's reasoning. Every major AI company (OpenAI, Anthropic, Google) logs and traces internally. Langfuse was built by engineers who realized every LLM app needs traces. Structured logs + traces are non-negotiable for production agents.

Distributed Tracing Instrumentation Cost Optimization Debugging
🎯 Mini Challenge: Add logging to an agent

Task: Take the agent from Lesson 18 (multi-turn chat). Add structured JSON logging for:

1. When an LLM call starts (model name, num messages)

2. When an LLM call ends (latency, token counts)

3. When a tool call starts/ends (tool name, result)

4. Any errors (exception message, context)

Run the agent and verify the logs are valid JSON on each line.

Hint: Use the JSONFormatter from structured_logging.py. Wrap your LLM call with logger.info(..., extra={...}). Test by piping output through `jq` or parsing in Python: `json.loads(line)`.

Test your knowledge

Question 1 of 5
What's the key advantage of structured JSON logs over free-text logs?
Question 2 of 5
What should you log for every LLM call?
Question 3 of 5
What is a trace in the context of agent debugging?
Question 4 of 5
Why is PII redaction important before logging?
Question 5 of 5
On a 3G network in Kenya, what does a trace help you diagnose?
0
/5
Excellent! You can now instrument and debug production agents.
Africa & India's Best AI Academy
🤖 AI-first 🎓 Placement Assistance

You've learned to see into agents.
Now deploy them with confidence.

Join the Best AI Program: The AI-Era Agentic Engineer — 5 months, fully online, with placement assistance. Week 1 is free.

Explore The AI-Era Agentic Engineer →