🎯 What you will learn
Prompts are the interface between your application and the LLM. Like any code, they break silently when changed, need version history, and must be tested before deployment. By the end of this lesson you will understand:
- Why prompts must be versioned in a registry (not scattered in String literals)
- The
PromptTemplaterecord with id, version, template, and params - Parameterised templates with
{placeholder}substitution viarender(Map) - Automated evaluation: keyword-presence assertions in JUnit 5 CI tests
- Version guard tests — fail CI when a template changes unexpectedly
- Advanced: LLM-as-judge evaluation pattern and A/B testing prompts
🎓 Real-world scenario
VidyaPath's "explain concept" feature broke silently after an accidental prompt change. Riya introduces a PromptTemplate registry with version IDs, parameterised placeholders, and automated evaluation tests — now prompt changes are tracked in Git and validated before deployment.
📋 PromptTemplate record + registry
The PromptTemplate record stores the template string, its version, and the list of expected placeholder names. The registry maps template IDs to their current versions — one source of truth for all prompts in the application.
Why a registry? When prompts are scattered as String literals across controllers and services, a change requires grep-searching the entire codebase. A centralised registry makes every prompt discoverable, versioned, and independently testable — the same discipline you apply to database schemas and API contracts.
🤖 Rendering + calling the LLM
The render(Map) method performs placeholder substitution. If any {placeholder} remains after substitution, it throws immediately — catching template bugs at call time rather than producing a broken prompt silently.
🧪 Automated prompt evaluation
LLM outputs are non-deterministic — you cannot assert exact string match. Instead, evaluate on soft criteria: required keywords present, response length in range, JSON valid (if expecting JSON), no forbidden words. Run tests at temperature=0 for maximum determinism.
LLM-as-judge: For richer evaluation, call a second LLM with the question, expected behaviour, and the generated response. Ask: "Does this response correctly explain the concept? Answer YES or NO with a one-line reason." Parse the YES/NO and log the reason. More reliable than keyword matching but costs an extra LLM call per evaluation.
📋 Prompt lifecycle workflow
| Phase | Action | Tool / Pattern |
|---|---|---|
| Author | Write template with {placeholders} | PromptTemplate record in REGISTRY |
| Version | Assign id + version (v1, v2, …) | Git commit for every version bump |
| Test | Keyword assertions in JUnit 5 | evaluate(response, mustContain) |
| Guard | Assert version hasn't changed unexpectedly | assertEquals("v2", template.version()) |
| Deploy | Merge after CI green on evaluation tests | GitHub Actions / Jenkins pipeline |
| A/B test | Route traffic across versions, measure quality | Feature flag + statistical significance test |
📝 Quiz — 5 questions
render(Map) method do?evaluate(response, mustContain) check?Next: Capstone — Spring Boot AI Service →
🎤 Interview Q&A
Prompt injection: a user crafts input that overrides your system prompt (e.g., "Ignore previous instructions and..."). Prevent it by: separating system instructions from user input using structured message roles (system/user/assistant), sanitising user input, and using a model that respects role separation.
Temperature (0.0–1.0) controls randomness. temperature=0 → deterministic (same prompt = same output, good for tests). temperature=0.7 → creative and varied (better for generation tasks). Set temperature=0 in prompt evaluation tests for reproducible results.
Deploy both v1 and v2 templates. Route 50% of traffic to each using a feature flag or random assignment. Collect user feedback and implicit signals (click-through, time-on-page). After N samples, run a statistical significance test (chi-square or t-test) and promote the winning version.
Store templates in a database table (prompt_templates: id, version, template, created_at) or a config server. Load at startup via a @Repository. Add a REST endpoint POST /admin/prompts/reload to hot-reload without restart.
Instead of checking for keywords, call a second LLM (the "judge") with the question, expected behavior, and the generated response. Ask: "Does this response correctly explain the concept? Answer YES or NO with a one-line reason." Parse the YES/NO and log the reason. More reliable than keyword matching but costs an extra LLM call per evaluation.
⚠️ Common mistakes
response.equals(expectedString) almost always fails for LLM output because the model varies phrasing. Use contains(), evaluate(), or LLM-as-judge.✅ Key takeaways
- Version prompts in a registry with id + version — track changes in Git like source code
- Parameterised templates (
{concept},{level}) separate structure from data - Automated evaluation: keyword checks in JUnit tests run in CI
- Version guard tests catch unexpected prompt changes before they reach production
- For richer evaluation: LLM-as-judge or embedding-similarity scoring
From prompt registries to a deployed AI service
Join the Best AI Programs — three 5-month courses, fully online, with placement assistance. Week 1 is free.
Explore our AI-era courses →