🎯 What you will learn

You don't need a heavyweight AI SDK to call a language model from Java. Ollama exposes a simple REST API — the same one curl uses. Java 11's built-in java.net.http.HttpClient is all you need. By the end of this lesson you will be able to:

  • POST to Ollama's /api/generate endpoint from plain Java
  • Use HttpClient, HttpRequest, and HttpResponse from java.net.http
  • Serialize a request body to JSON with Jackson's ObjectMapper
  • Parse the JSON response into a JsonNode tree
  • GET /api/tags to list all locally installed models
  • Understand the difference between /api/generate and /api/chat
💼 Interview reality: "How would you integrate an LLM into a Java backend without a framework?" is asked in AI-focused Java rounds. Showing you can do it with zero third-party HTTP libraries (only Jackson for JSON) is a strong signal.

🎓 Real-world scenario

🎓
VidyaPath · Bangalore, India
LLM-powered Java tutor — no SDK required

VidyaPath adds AI-generated concept explanations to its platform. Rather than importing a heavy AI SDK, the team uses Java 11+'s built-in HttpClient to POST directly to Ollama's REST API — the same HTTP call any curl command would make.

🌐 Basic Ollama HTTP call

The OllamaClient class holds the HTTP client (singleton, thread-safe) and an ObjectMapper for JSON. The generate method builds a request with Map.of(), serializes it, sends it, and extracts the "response" field from the returned JSON.

javac -cp "jackson-core.jar:jackson-databind.jar:." OllamaClient.java
OllamaClient.javaLesson 34 · HttpClient + Ollama /api/generate
verified output
=== VidyaPath — Calling an LLM from Java ===

Why stream=false? By default Ollama streams tokens as they are generated (Server-Sent Events / NDJSON). Setting "stream": false tells Ollama to wait until the model finishes, then return one complete JSON object. Simpler to parse for one-shot use cases.

💬 Factual Q&A with llama3.2:3b

The main method calls generate twice — once for a factual question and once for code generation. Both are simple prompt strings; the model decides the format of the response.

java -cp ".:jackson-core.jar:jackson-databind.jar" OllamaClient
OllamaClient.java — main()Lesson 34 · Q&A + code generation
verified output — captured from llama3.2:3b
=== VidyaPath — Calling an LLM from Java === Q: What is Java's garbage collector? A: Java's garbage collector is a component of the Java Runtime Environment (JRE) that automatically frees up memory occupied by objects that are no longer needed or referenced in the application, thereby reducing memory waste and improving performance. The garbage collector uses various algorithms and data structures to identify eligible objects for destruction, which can be configured through parameters such as heap size and garbage collection frequency. --- Code generation --- ```java public static int fibonacci(int n) { Map<Integer, Integer> memo = new HashMap<>(); return fib(n, memo); } private static int fib(int n, Map<Integer, Integer> memo) { if (n < 2) return n; if (memo.containsKey(n)) return memo.get(n); int res = fib(n-1, memo) + fib(n-2, memo); memo.put(n, res); return res; } ```

📋 List available models via /api/tags

Before sending a prompt, it's good practice to verify the model exists locally. A simple GET /api/tags returns a JSON array of installed models with their sizes and parameter counts.

java -cp ".:jackson-core.jar:jackson-databind.jar" OllamaClient
OllamaClient.java — listModels()Lesson 34 · GET /api/tags + API reference
verified output — from running Ollama instance
--- Available models --- nomic-embed-text:latest deepseek-r1:7b mistral:7b phi3.5:3.8b gemma2:2b llama3.2:3b

📋 Ollama API endpoints at a glance

MethodEndpointPurposeKey body fields
POST/api/generateSingle prompt completionmodel, prompt, stream
POST/api/chatMulti-turn chat (messages array)model, messages, stream
POST/api/embedGenerate vector embeddings (RAG)model, input
GET/api/tagsList installed models—
GET/api/showModel details (context length, params)name (query param)

🎤 Interview Q&A

EasyWhat is Ollama and why use it for development?
+

Ollama is a local LLM server that runs open models (llama3, mistral, phi, gemma etc.) on your machine. No API key, no cost, no data sent to the cloud — ideal for development and privacy-sensitive applications. The REST API is almost identical to OpenAI's, so switching to a cloud provider later is straightforward.

EasyWhat is the difference between /api/generate and /api/chat?
+

/api/generate is for single-turn prompts: you send one string. /api/chat takes a messages array like OpenAI's Chat API format: [{"role":"user","content":"..."}, {"role":"assistant","content":"..."}] — for multi-turn conversation with history. Use /api/generate for simple one-shot tasks; /api/chat for chatbots that maintain context.

MediumHow do you handle streaming responses from Ollama?
+

Set stream=true in the request. Ollama returns newline-delimited JSON objects (NDJSON). Use HttpResponse.BodyHandlers.ofLines() to get a Stream<String>; parse each line and extract "response" until "done":true. Each line is a partial token — concatenate them for the full response. This approach lets you forward tokens to a browser via SSE in real time.

MediumWhat models work best for code generation vs. factual Q&A?
+

For code: deepseek-r1, codellama, phi3.5. For general Q&A: llama3.2, mistral, gemma2. Smaller models (3B–7B) run well on a laptop GPU or even CPU. Larger models (13B+) need a GPU with 8GB+ VRAM. For a tutor application, llama3.2:3b gives a good balance of quality and speed on a 16GB MacBook.

HardHow would you implement retry logic and timeout for Ollama calls?
+

Set .timeout(Duration.ofSeconds(60)) on HttpRequest for per-request timeout. For retry: use a loop with exponential backoff — Thread.sleep(Math.pow(2, attempt) * 1000) — catching IOException and HttpTimeoutException. Or use Resilience4j's @Retry annotation in Spring Boot for declarative retry with configurable max attempts and backoff strategy.

📝 Quiz — 5 questions

Question 1 of 5
Which Java class is used to make HTTP requests in modern Java (11+)?
Question 2 of 5
What does "stream": false mean in the Ollama request body?
Question 3 of 5
How do you parse JSON in Java without a framework?
Question 4 of 5
What does HttpRequest.BodyPublishers.ofString(body) do?
Question 5 of 5
What Ollama endpoint lists all locally installed models?
0
/ 5
Quiz complete!

Next: Spring Boot AI Gateway →

⚠️ Common mistakes

Forgetting stream=false
Without stream=false, Ollama returns streaming NDJSON lines. Parsing the body as a single JSON object will fail with a parse error because the response is multiple lines, each a partial JSON object.
Not handling large responses
The Ollama response can be several KB. HttpResponse.BodyHandlers.ofString() buffers the whole response in memory — fine for small models but be mindful for large contexts (32k+ tokens).
Hardcoding the model name
Different models have different strengths. Inject the model name via @Value("${ollama.default-model}") in Spring Boot so you can switch models in config without code changes.
Missing Content-Type header
Without Content-Type: application/json Ollama returns a 400 error or parses the body incorrectly. Always set it — .header("Content-Type", "application/json") — before calling .POST().

✅ Key takeaways

  • java.net.http.HttpClient (Java 11+) makes REST calls with no extra dependencies
  • Ollama's POST /api/generate accepts model + prompt; set stream=false for simple use
  • Jackson's ObjectMapper.readTree() parses the JSON response into a traversable tree
  • GET /api/tags lists all installed models — use to validate availability at startup
  • Abstract the LLM call into a service class — swap Ollama for OpenAI/Anthropic in one place
Africa & India's Best AI Academy
🤖 AI-first 🎓 Placement Assistance

From HttpClient to a production AI service

Join the Best AI Programs — three 5-month courses, fully online, with placement assistance. Week 1 is free.

Explore our AI-era courses →