🎯 What you will learn
In the previous lesson we called Ollama directly using bare java.net.http.HttpClient. Now we wrap that logic behind a proper Spring Boot REST API — a pattern called an AI gateway. By the end of this lesson you will be able to:
- Use Spring 6.1's
RestClientfor outbound HTTP calls to Ollama - Build an
OllamaService@Servicewith a cleangenerate()method - Expose
POST /api/ai/chatandPOST /api/ai/explainvia@RestController - Use Java records as request/response DTOs — auto-serialized by Jackson
- Store a prompt history list and expose it at
GET /api/ai/history - Use
@RequestParamwithdefaultValuefor parameterised prompt templates
🎓 Real-world scenario
VidyaPath's web app, mobile app, and backend analytics all need LLM access. Instead of each calling Ollama directly, Riya builds a single Spring Boot gateway: centralised prompt logging, model selection, request history, and a clean REST contract that hides the Ollama API from downstream clients.
🔧 OllamaService with RestClient
Spring 6.1 introduced RestClient — a synchronous, fluent HTTP client that replaces the deprecated RestTemplate. It reads cleaner than RestTemplate while staying blocking (unlike the reactive WebClient). The service returns a ChatResponse record that includes timing metadata.
Record DTOs: record ChatResponse(String model, String prompt, String response, long durationMs, LocalDateTime timestamp) is a concise immutable DTO. Jackson serialises it automatically — no @JsonProperty or getters needed. Similarly, record ChatRequest(String prompt, String model) deserialises inbound JSON.
🌐 AiController + chat endpoint
The controller is a thin layer: it delegates to OllamaService, defaults the model if the caller omits it, stores every response in an in-memory history list, and exposes three endpoints. Jackson serialises the record response to JSON automatically.
💡 Explain endpoint with prompt template
The /api/ai/explain endpoint demonstrates parameterised prompt templates: the topic and level come from query params, the gateway builds the full prompt string, and the LLM never sees raw user input unformatted. This is the simplest form of prompt engineering — a preview of Lesson 37.
📋 Gateway endpoints at a glance
| Method | Endpoint | Purpose | Body / Params |
|---|---|---|---|
POST | /api/ai/chat | Send a prompt, get a response | JSON: {"prompt":"...", "model":"..."} |
POST | /api/ai/explain | Explain a topic at a skill level | Query: ?topic=...&level=beginner |
GET | /api/ai/models | List installed Ollama models | — |
GET | /api/ai/history | Retrieve all past prompts + responses | — |
🎤 Interview Q&A
record ChatRequest(String prompt, String model) auto-generates constructor, getters, equals, hashCode, toString. For DTOs that are pure data carriers with no behaviour, records are cleaner and more concise than full classes. Jackson deserialises JSON into records via the canonical constructor — no @JsonCreator annotation needed in Spring Boot 3+.
Use a bounded deque: Deque<ChatResponse> history = new ArrayDeque<>(1000) — or better, persist to a database and query with pagination. For a true gateway, use Redis or a time-series DB for analytics. An in-memory list is fine for demos but loses all history on restart.
Extract an LlmService interface with a generate(model, prompt) method. Implement OllamaService and OpenAiService. Bind them to @Profile("local") and @Profile("cloud") respectively. Switch with SPRING_PROFILES_ACTIVE=cloud in prod. The controller only knows about LlmService — the backend is invisible to it.
Use Bucket4j (token bucket algorithm) or Spring Cloud Gateway's rate filter. Inject a RateLimiter bean and check it before calling ollamaService.generate(). Return 429 Too Many Requests if the limit is exceeded. For per-user limits, key the bucket by JWT subject or IP address extracted from the request.
Switch RestClient (blocking) to WebClient (reactive). Change the service return type to Mono<ChatResponse> and the controller to return Mono<ChatResponse>. Spring WebFlux handles I/O-bound waits without holding threads. Each concurrent request waits on the event loop, not a blocked thread — supporting hundreds of concurrent LLM calls on a small server.
📝 Quiz — 5 questions
restClient.post().uri(url).body(map).retrieve().body(JsonNode.class) do?Next: RAG with Java and Postgres →
⚠️ Common mistakes
ArrayList in the controller is shared across all requests. Concurrent requests can corrupt it. Use CopyOnWriteArrayList or (better) persist to a database table with a timestamp column.durationMs at INFO level for every LLM call.RestClient will throw RestClientResponseException. Catch it in @RestControllerAdvice and return a friendly error JSON instead of an unhandled 500.✅ Key takeaways
- RestClient (Spring 6.1) is the modern synchronous HTTP client for calling external APIs
- Wrap Ollama behind a @Service — swap the backend without touching controllers
- Record DTOs (ChatRequest, ChatResponse) map to JSON cleanly with zero config
- Centralise history, rate limiting, and model selection in the gateway
- For async/high throughput: replace RestClient + @Service with WebClient + reactive pipeline
From AI Gateway to full RAG pipeline
Join the Best AI Programs — three 5-month courses, fully online, with placement assistance. Week 1 is free.
Explore our AI-era courses →