🎯 What you will learn

In the previous lesson we called Ollama directly using bare java.net.http.HttpClient. Now we wrap that logic behind a proper Spring Boot REST API — a pattern called an AI gateway. By the end of this lesson you will be able to:

  • Use Spring 6.1's RestClient for outbound HTTP calls to Ollama
  • Build an OllamaService @Service with a clean generate() method
  • Expose POST /api/ai/chat and POST /api/ai/explain via @RestController
  • Use Java records as request/response DTOs — auto-serialized by Jackson
  • Store a prompt history list and expose it at GET /api/ai/history
  • Use @RequestParam with defaultValue for parameterised prompt templates
💼 Interview reality: "How do you hide a third-party AI API from downstream clients?" is a common system design question. The gateway pattern — a thin Spring Boot service that owns the LLM contract — is the standard answer.

🎓 Real-world scenario

🎓
VidyaPath · Bangalore, India
One AI gateway for every client

VidyaPath's web app, mobile app, and backend analytics all need LLM access. Instead of each calling Ollama directly, Riya builds a single Spring Boot gateway: centralised prompt logging, model selection, request history, and a clean REST contract that hides the Ollama API from downstream clients.

🔧 OllamaService with RestClient

Spring 6.1 introduced RestClient — a synchronous, fluent HTTP client that replaces the deprecated RestTemplate. It reads cleaner than RestTemplate while staying blocking (unlike the reactive WebClient). The service returns a ChatResponse record that includes timing metadata.

mvn spring-boot:run
OllamaService.javaLesson 35 · RestClient + @Service
startup output
:: Spring Boot :: (v3.3.5) Started VidyaPathApplication in 0.568 seconds (process running for 0.667)

Record DTOs: record ChatResponse(String model, String prompt, String response, long durationMs, LocalDateTime timestamp) is a concise immutable DTO. Jackson serialises it automatically — no @JsonProperty or getters needed. Similarly, record ChatRequest(String prompt, String model) deserialises inbound JSON.

🌐 AiController + chat endpoint

The controller is a thin layer: it delegates to OllamaService, defaults the model if the caller omits it, stores every response in an in-memory history list, and exposes three endpoints. Jackson serialises the record response to JSON automatically.

curl -s -X POST http://localhost:8767/api/ai/chat -H "Content-Type: application/json" -d '{"prompt":"What is Spring Boot in one sentence?","model":"llama3.2:3b"}'
AiController.javaLesson 35 · @RestController + history
verified output — from running gateway + llama3.2:3b{ "model": "llama3.2:3b", "prompt": "What is Spring Boot in one sentence?", "response": "Spring Boot is a popular open-source framework that simplifies the process of building web applications and microservices using the Java programming language, providing a concise and efficient way to create robust, scalable, and production-ready applications.", "durationMs": 1403, "timestamp": "2026-04-26T21:11:02.498952" }

💡 Explain endpoint with prompt template

The /api/ai/explain endpoint demonstrates parameterised prompt templates: the topic and level come from query params, the gateway builds the full prompt string, and the LLM never sees raw user input unformatted. This is the simplest form of prompt engineering — a preview of Lesson 37.

curl -s -X POST "http://localhost:8767/api/ai/explain?topic=Java+streams&level=beginner"
AiController.java — explain()Lesson 35 · @RequestParam + prompt template
verified output — truncated, captured from llama3.2:3b{ "model": "llama3.2:3b", "prompt": "Explain 'Java streams' to a beginner Java developer...", "response": "Here are three bullet points explaining Java streams:\n\n • What is a Stream?: A stream is a sequence of elements that can be processed in a pipeline-like fashion...\n\n • How do Streams Work?: You start by selecting a collection and then specify operations using methods like filter(), map(), and sorted().\n\n • Example: Finding all Employees Over 30: employees.stream()\n .filter(e -> e.getAge() > 30).collect(Collectors.toList())", "durationMs": 10530, "timestamp": "2026-04-26T21:10:57.797771" }

📋 Gateway endpoints at a glance

MethodEndpointPurposeBody / Params
POST/api/ai/chatSend a prompt, get a responseJSON: {"prompt":"...", "model":"..."}
POST/api/ai/explainExplain a topic at a skill levelQuery: ?topic=...&level=beginner
GET/api/ai/modelsList installed Ollama models—
GET/api/ai/historyRetrieve all past prompts + responses—

🎤 Interview Q&A

EasyWhat is a ChatRequest record and why use a record instead of a class?
+

record ChatRequest(String prompt, String model) auto-generates constructor, getters, equals, hashCode, toString. For DTOs that are pure data carriers with no behaviour, records are cleaner and more concise than full classes. Jackson deserialises JSON into records via the canonical constructor — no @JsonCreator annotation needed in Spring Boot 3+.

EasyHow do you prevent the history list from growing indefinitely in production?
+

Use a bounded deque: Deque<ChatResponse> history = new ArrayDeque<>(1000) — or better, persist to a database and query with pagination. For a true gateway, use Redis or a time-series DB for analytics. An in-memory list is fine for demos but loses all history on restart.

MediumHow do you switch between Ollama and OpenAI's API without changing the controller?
+

Extract an LlmService interface with a generate(model, prompt) method. Implement OllamaService and OpenAiService. Bind them to @Profile("local") and @Profile("cloud") respectively. Switch with SPRING_PROFILES_ACTIVE=cloud in prod. The controller only knows about LlmService — the backend is invisible to it.

MediumHow would you add rate limiting to the /api/ai/chat endpoint?
+

Use Bucket4j (token bucket algorithm) or Spring Cloud Gateway's rate filter. Inject a RateLimiter bean and check it before calling ollamaService.generate(). Return 429 Too Many Requests if the limit is exceeded. For per-user limits, key the bucket by JWT subject or IP address extracted from the request.

HardHow would you make the gateway non-blocking to handle 100 concurrent LLM requests?
+

Switch RestClient (blocking) to WebClient (reactive). Change the service return type to Mono<ChatResponse> and the controller to return Mono<ChatResponse>. Spring WebFlux handles I/O-bound waits without holding threads. Each concurrent request waits on the event loop, not a blocked thread — supporting hundreds of concurrent LLM calls on a small server.

📝 Quiz — 5 questions

Question 1 of 5
What is Spring 6.1's modern HTTP client for outbound calls?
Question 2 of 5
What Java record is used as the response DTO in this lesson?
Question 3 of 5
What does restClient.post().uri(url).body(map).retrieve().body(JsonNode.class) do?
Question 4 of 5
Why keep a history list in the controller?
Question 5 of 5
What is the advantage of a centralized AI gateway over each service calling Ollama directly?
0
/ 5
Quiz complete!

Next: RAG with Java and Postgres →

⚠️ Common mistakes

Storing state in a singleton @Service
The history ArrayList in the controller is shared across all requests. Concurrent requests can corrupt it. Use CopyOnWriteArrayList or (better) persist to a database table with a timestamp column.
Not logging prompt + model + duration
Without structured logging, you can't debug why responses are slow or wrong. Always log model name, prompt hash, response length, and durationMs at INFO level for every LLM call.
Ignoring HTTP error responses from Ollama
If Ollama returns a 404 (model not found) or 500, RestClient will throw RestClientResponseException. Catch it in @RestControllerAdvice and return a friendly error JSON instead of an unhandled 500.
Passing raw user input directly to the LLM
Production gateways need prompt injection checks. Never concatenate raw user input into system prompts without sanitisation — users can attempt prompt injection attacks that override your instructions.

✅ Key takeaways

  • RestClient (Spring 6.1) is the modern synchronous HTTP client for calling external APIs
  • Wrap Ollama behind a @Service — swap the backend without touching controllers
  • Record DTOs (ChatRequest, ChatResponse) map to JSON cleanly with zero config
  • Centralise history, rate limiting, and model selection in the gateway
  • For async/high throughput: replace RestClient + @Service with WebClient + reactive pipeline
Africa & India's Best AI Academy
🤖 AI-first 🎓 Placement Assistance

From AI Gateway to full RAG pipeline

Join the Best AI Programs — three 5-month courses, fully online, with placement assistance. Week 1 is free.

Explore our AI-era courses →