🎯 What you will build

This capstone builds a four-step LLM pipeline in plain Java: generate → evaluate → index → serve. By the end of this lesson you will have shipped:

  • A generate() step that calls Ollama for a 2-sentence technical summary per topic
  • An evaluate() quality gate using keyword presence — prevents garbage from entering the knowledge base
  • An index() step that writes approved summaries to an in-memory List<KbEntry> (swap for PostgreSQL + pgvector in production)
  • A search() step that retrieves entries by keyword match over topic and summary text
  • A complete main() that runs the pipeline for 3 Java topics and prints a summary report
🏆 Pipeline thinking: Every production ML/AI system has this shape: Generate → Evaluate → Index → Serve. Understanding these four stages — and what can go wrong at each — is what distinguishes senior engineers from junior ones.

🏆 Real-world scenario

🏆
VidyaPath · Bangalore, India
Full LLM pipeline from ingestion to retrieval

VidyaPath needs to auto-generate and quality-check explanations for 500 Java topics. Riya builds a pipeline: for each topic, call Ollama to generate a summary, run an automated keyword-quality gate, index approved summaries, and serve them through a search endpoint. Three lessons (34, 36, 37) in one runnable program.

🏗️ Block 1 — Pipeline architecture + records

The pipeline is a single class with four static methods — one per stage. Java records make the data shapes immutable and concise. The in-memory knowledgeBase list is a stand-in for PostgreSQL + pgvector in production.

javac -cp ".:jackson-core.jar:jackson-databind.jar" LlmPipeline.java
LlmPipeline.java — architecture + recordsLesson 39 · records, List, static methods
compilation output
Compiled successfully — LLM pipeline ready

Records as pipeline contracts: PipelineResult and KbEntry are Java 16+ records — immutable, auto-generated equals/hashCode/toString, zero boilerplate. Each pipeline stage returns a record, making the data flow explicit and testable.

⚙️ Block 2 — Running the pipeline

runPipeline() orchestrates all four steps. The quality gate threshold is matched.size() >= required.size() / 2 — at least half the keywords must appear. Approved topics are indexed; rejected ones are counted in the summary report.

java -cp ".:jackson-core.jar:jackson-databind.jar" LlmPipeline
LlmPipeline.java — runPipeline + mainLesson 39 · pipeline orchestration, Map.entry
verified output — pipeline processing 3 topics
=== VidyaPath — LLM Pipeline === Processing: Java generics ... PASS ✓ Processing: Spring Boot ... PASS ✓ Processing: JUnit 5 ... PASS ✓ === Pipeline Summary === Processed : 3 Approved : 3 Rejected : 0

Quality gate threshold: The condition matched.size() >= required.size() / 2 requires at least half the expected keywords. For production, replace this with an LLM-as-judge call: ask llama3.2:3b to rate the summary quality 1–5 and reject anything below 3.

🎓 Complete Java Track — What You've Shipped

🗺️ 8 Arcs · 39 Lessons · One Full-Stack AI Engineer

Arc A · 00–07
Core Java syntax — variables, operators, control flow, methods, OOP foundations
Arc B · 08–13
Object-Oriented Java — classes, constructors, inheritance, polymorphism, encapsulation
Arc C · 14–18
Modern Java — collections, generics, lambdas, streams, exceptions
Arc D · 19–26
Spring Boot REST API — controllers, DI, validation, error handling, security, config
Arc E · 27–30
Databases — JDBC, JPA, relationships, @Transactional
Arc F · 31–33
Testing — JUnit 5, Mockito, Testcontainers
Arc G · 34–37
Java + AI — calling LLMs, Spring AI gateway, RAG, prompt versioning
Arc H · 38–39
Capstones — full-stack Spring Boot AI service + LLM pipeline

🎤 Interview Q&A

EasyWhat is a microservice and why split this pipeline into one?
+

A microservice is a small, independently deployable service with a single responsibility. This pipeline could be its own service: receive topics via Kafka, generate summaries, and write to the knowledge DB. Other services (search, chat) consume from the knowledge DB without coupling to generation logic. Each service scales independently — scale the generator separately from the search service under load.

EasyWhat is the difference between sequential and parallel pipeline processing?
+

Sequential: each topic finishes before the next starts. Parallel: multiple topics processed simultaneously. For 500 topics × 5s each: sequential = 2,500s; parallel (50 threads) = ~50s. Use virtual threads (Thread.ofVirtual().start(() -> runPipeline(...))) or CompletableFuture.allOf() for parallelism. Virtual threads are the Java 21 way — no thread pool exhaustion.

MediumHow would you handle pipeline failures — partial indexing on error?
+

Add a @Transactional boundary around the full pipeline run for a batch. If any topic fails, the entire batch rolls back. Or: use a status column (PENDING/INDEXED/FAILED) in the DB. Process PENDING topics, mark INDEXED on success, FAILED on error — retry FAILED entries later with exponential backoff. This approach is idempotent: re-running processes only un-indexed topics.

MediumWhat is a dead-letter queue in the context of this pipeline?
+

If a topic fails repeatedly (Ollama timeout, quality gate never passes), a dead-letter queue (DLQ) catches it after N retries. In Kafka: configure a DLQ topic; failed messages land there for manual review. This prevents one bad topic from blocking the whole pipeline. Engineers subscribe to DLQ alerts and investigate whether the topic itself is problematic (too obscure for the model) or the system is broken.

HardHow would you make this pipeline idempotent?
+

Idempotent: running the pipeline twice for the same topic produces the same result with no duplicate entries. Implement with: (1) generate a deterministic ID for each topic (e.g., SHA-256 of topic string); (2) use INSERT ... ON CONFLICT DO NOTHING in PostgreSQL; (3) check EXISTS before generating/indexing. This makes the pipeline safe to retry after partial failures — a property production systems must have.

📝 Quiz — 5 questions

Question 1 of 5
What is the purpose of the evaluation step in the pipeline?
Question 2 of 5
What would you replace the in-memory List with in production?
Question 3 of 5
Which Java 21 feature would let you parallelise the pipeline steps for 500 topics efficiently?
Question 4 of 5
How would you schedule the pipeline to run nightly?
Question 5 of 5
What JPA annotation would you add to make the indexing step transactional?
0
/ 5
Java Track complete! 🎉

Next: Java Cheatsheet — Syntax Reference →

⚠️ Common mistakes

No quality gate — indexing every LLM response
LLMs sometimes hallucinate or produce off-topic responses. A quality gate (keyword check, LLM-as-judge, or minimum length) prevents garbage from entering the knowledge base and corrupting RAG search results.
Blocking main thread with sequential LLM calls
For large topic lists, sequential processing is extremely slow. Use CompletableFuture.supplyAsync() with a bounded thread pool or virtual threads (Thread.ofVirtual()) to process multiple topics concurrently.
Mutable shared state in parallel pipelines
A static List<KbEntry> knowledgeBase accessed concurrently without synchronisation causes race conditions. Use CopyOnWriteArrayList, ConcurrentLinkedQueue, or synchronise on the database (one writer, many readers).
No observability on the pipeline
Without logging each topic's status, duration, and evaluation score, debugging a failing pipeline is guesswork. Log: topic, LLM duration, evaluation score, PASS/FAIL, and any exception. Use structured JSON logs for aggregation.

✅ Key takeaways

  • LLM pipeline = Generate → Evaluate → Index → Serve — four clear, testable steps
  • Quality gates prevent bad LLM output from polluting the knowledge base
  • Replace the in-memory List with PostgreSQL + pgvector for production semantic search
  • Virtual threads (Java 21) parallelise LLM calls without exhausting OS thread pools
  • Add @Transactional + @Scheduled in Spring Boot to make the pipeline managed and schedulable
Africa & India's Best AI Academy
🤖 AI-first 🎓 Placement Assistance

You've built a production-ready Java AI stack

Join the Best AI Programs — three 5-month courses, fully online, with placement assistance. Week 1 is free.

Explore our AI-era courses →