🎯 What you will build
This capstone builds a four-step LLM pipeline in plain Java: generate → evaluate → index → serve. By the end of this lesson you will have shipped:
- A
generate()step that calls Ollama for a 2-sentence technical summary per topic - An
evaluate()quality gate using keyword presence — prevents garbage from entering the knowledge base - An
index()step that writes approved summaries to an in-memoryList<KbEntry>(swap for PostgreSQL + pgvector in production) - A
search()step that retrieves entries by keyword match over topic and summary text - A complete
main()that runs the pipeline for 3 Java topics and prints a summary report
🏆 Real-world scenario
VidyaPath needs to auto-generate and quality-check explanations for 500 Java topics. Riya builds a pipeline: for each topic, call Ollama to generate a summary, run an automated keyword-quality gate, index approved summaries, and serve them through a search endpoint. Three lessons (34, 36, 37) in one runnable program.
🏗️ Block 1 — Pipeline architecture + records
The pipeline is a single class with four static methods — one per stage. Java records make the data shapes immutable and concise. The in-memory knowledgeBase list is a stand-in for PostgreSQL + pgvector in production.
Records as pipeline contracts: PipelineResult and KbEntry are Java 16+ records — immutable, auto-generated equals/hashCode/toString, zero boilerplate. Each pipeline stage returns a record, making the data flow explicit and testable.
⚙️ Block 2 — Running the pipeline
runPipeline() orchestrates all four steps. The quality gate threshold is matched.size() >= required.size() / 2 — at least half the keywords must appear. Approved topics are indexed; rejected ones are counted in the summary report.
Quality gate threshold: The condition matched.size() >= required.size() / 2 requires at least half the expected keywords. For production, replace this with an LLM-as-judge call: ask llama3.2:3b to rate the summary quality 1–5 and reject anything below 3.
🔍 Block 3 — Knowledge base + RAG search
After the pipeline runs, the knowledge base holds all approved summaries. The search() method performs substring matching over both topic and summary text — a simple but functional RAG retrieval layer. In production, this becomes a vector cosine similarity search using pgvector.
From keyword to vector search: The search() method works with string contains — fast and simple. To find semantically similar topics (e.g. "generic types" returning both "Java generics" and "type bounds"), generate embeddings with Ollama's /api/embed endpoint and store them in a pgvector column. Cosine similarity search then replaces substring matching entirely.
🎓 Complete Java Track — What You've Shipped
🗺️ 8 Arcs · 39 Lessons · One Full-Stack AI Engineer
🎤 Interview Q&A
A microservice is a small, independently deployable service with a single responsibility. This pipeline could be its own service: receive topics via Kafka, generate summaries, and write to the knowledge DB. Other services (search, chat) consume from the knowledge DB without coupling to generation logic. Each service scales independently — scale the generator separately from the search service under load.
Sequential: each topic finishes before the next starts. Parallel: multiple topics processed simultaneously. For 500 topics × 5s each: sequential = 2,500s; parallel (50 threads) = ~50s. Use virtual threads (Thread.ofVirtual().start(() -> runPipeline(...))) or CompletableFuture.allOf() for parallelism. Virtual threads are the Java 21 way — no thread pool exhaustion.
Add a @Transactional boundary around the full pipeline run for a batch. If any topic fails, the entire batch rolls back. Or: use a status column (PENDING/INDEXED/FAILED) in the DB. Process PENDING topics, mark INDEXED on success, FAILED on error — retry FAILED entries later with exponential backoff. This approach is idempotent: re-running processes only un-indexed topics.
If a topic fails repeatedly (Ollama timeout, quality gate never passes), a dead-letter queue (DLQ) catches it after N retries. In Kafka: configure a DLQ topic; failed messages land there for manual review. This prevents one bad topic from blocking the whole pipeline. Engineers subscribe to DLQ alerts and investigate whether the topic itself is problematic (too obscure for the model) or the system is broken.
Idempotent: running the pipeline twice for the same topic produces the same result with no duplicate entries. Implement with: (1) generate a deterministic ID for each topic (e.g., SHA-256 of topic string); (2) use INSERT ... ON CONFLICT DO NOTHING in PostgreSQL; (3) check EXISTS before generating/indexing. This makes the pipeline safe to retry after partial failures — a property production systems must have.
📝 Quiz — 5 questions
List with in production?Next: Java Cheatsheet — Syntax Reference →
⚠️ Common mistakes
CompletableFuture.supplyAsync() with a bounded thread pool or virtual threads (Thread.ofVirtual()) to process multiple topics concurrently.static List<KbEntry> knowledgeBase accessed concurrently without synchronisation causes race conditions. Use CopyOnWriteArrayList, ConcurrentLinkedQueue, or synchronise on the database (one writer, many readers).✅ Key takeaways
- LLM pipeline = Generate → Evaluate → Index → Serve — four clear, testable steps
- Quality gates prevent bad LLM output from polluting the knowledge base
- Replace the in-memory
Listwith PostgreSQL + pgvector for production semantic search - Virtual threads (Java 21) parallelise LLM calls without exhausting OS thread pools
- Add
@Transactional+@Scheduledin Spring Boot to make the pipeline managed and schedulable
You've built a production-ready Java AI stack
Join the Best AI Programs — three 5-month courses, fully online, with placement assistance. Week 1 is free.
Explore our AI-era courses →