What is OpenSearch?
OpenSearch is a community fork of Elasticsearch, released under the Apache 2.0 licence. It is powered by Lucene under the hood and exposes a full-text search engine plus analytics capabilities over an HTTP REST API. The fork was created in 2021 when Elastic changed its licence terms; AWS and the open-source community maintained the Apache 2.0 fork as OpenSearch.
Used for: log analytics (EFK/ELK-style stacks), e-commerce product search, application monitoring, and security event analytics. In this track's Docker Compose setup, OpenSearch 2.x runs with security disabled for local development — no credentials needed on localhost:9200.
Tunde, a data engineer at NaijaCart, uses OpenSearch to power the product search bar that serves millions of product queries per day across Lagos, Abuja, and Port Harcourt. This lesson walks through the exact API calls he uses every day — from creating an index to running aggregations.
OpenSearch vs Cassandra vs Elasticsearch
All three are popular in big data stacks. Understanding when to pick OpenSearch versus Cassandra is a common interview topic.
| Dimension | OpenSearch | Cassandra |
|---|---|---|
| Primary use | Full-text search, log analytics | Time-series, write-heavy workloads |
| Query | Lucene full-text + aggregations | CQL (SQL-like, limited) |
| Licence | Apache 2.0 (OpenSearch) / Elastic Licence (ES) | Apache 2.0 |
| Schema | Dynamic (auto-detect types) | Strict (schema-first) |
| Data model | Documents (JSON) | Wide-column rows |
Health check
Before running any API calls, verify OpenSearch is up and healthy. Start your Docker stack with ./start.sh, then run:
OpenSearch must be running at localhost:9200. Output will vary.
status: "green" means all primary and replica shards are assigned. On a single-node cluster with number_of_replicas: 0, this is the expected healthy state. yellow means replicas are unassigned — normal on single-node if you accidentally set replicas to 1.
Create an index
An index in OpenSearch is analogous to a table in a relational database. Wanjiku created the naijacart-products index with explicit field mappings so that OpenSearch does not guess types incorrectly.
OpenSearch must be running at localhost:9200. Output will vary.
text fields are tokenised and analysed — perfect for full-text search (e.g., searching "5000mAh battery" inside a description). keyword fields are stored as-is — perfect for exact match filters and aggregations (e.g., filtering by category = "Electronics"). Using text on a field you want to aggregate is a common mistake covered in the mistakes section below.
Index a document
Indexing a document is the OpenSearch equivalent of inserting a row. The _doc endpoint accepts a JSON body. OpenSearch assigns an auto-generated _id if you do not specify one.
OpenSearch must be running at localhost:9200. Output will vary.
Full-text search
The match query tokenises the search string and finds documents whose text fields contain those tokens. This is where OpenSearch shines — the relevance score (_score) ranks results by how closely they match.
OpenSearch must be running at localhost:9200. Output will vary.
The _score value is computed using BM25 (Best Match 25), the default Lucene relevance algorithm. A higher score means a stronger match. Chidi at NaijaCart uses match queries for the search bar and term queries for exact category filtering.
Filter + range query
The bool query combines multiple clauses. Using filter instead of must means OpenSearch skips relevance scoring — filters are cached and significantly faster for exact-match and range conditions.
OpenSearch must be running at localhost:9200. Output will vary.
Aggregation — terms bucket
Aggregations are OpenSearch's analytics engine. A terms aggregation groups documents by a keyword field value — like GROUP BY in SQL. The avg aggregation computes a numeric average across all matched documents.
OpenSearch must be running at localhost:9200. Output will vary.
size: 0 mean?Setting size: 0 at the top level tells OpenSearch to return zero documents in the hits array — only the aggregation results are returned. This is a best practice for aggregation-only queries: it reduces response size and avoids Lucene fetching document source unnecessarily.
Python client
The opensearch-py library wraps the HTTP REST API with a Pythonic interface. Install it with pip install opensearch-py==2.6.0 (already included in the Lesson 00 pip install block).
The Python client is used in later lessons where OpenSearch is the consumer destination for Kafka events. For development, the curl commands above are easier to iterate on quickly.
Chidi's search tip
"At NaijaCart, I use text type for product descriptions — they need full-text tokenisation so a search for '5000mAh' finds '5000mAh battery'. I use keyword for category and city — exact match and aggregatable without any sub-field tricks. Never use text for fields you want to aggregate — it won't work without a .keyword sub-field, and dynamic mapping sometimes adds one silently. Explicit mappings eliminate the confusion."
Common mistakes
text type for aggregationskeyword fields. A text field is tokenised and cannot be bucketed without a .keyword sub-field. Always declare keyword type for category, city, status, and other exact-value fields.number_of_shards: 1 in local dev. Scale shards when you actually need horizontal distribution.number_of_replicas: 0yellow (unassigned replicas) forever. Always set number_of_replicas: 0 for local single-node development.Quiz — 3 questions
size: 0 at the top level of an OpenSearch search request mean?Next: pipe Kafka events into OpenSearch in real time.
Interview Q&A
bool query, a must (or should) clause contributes to relevance scoring — documents that match the clause score higher. A filter clause is a binary yes/no check: the document either matches or it does not, and it does not affect the _score. The performance implication is significant: filter clauses are cached by OpenSearch at the segment level in a bitset (bitfield of matching document IDs). Subsequent queries with the same filter retrieve the cached bitset instead of re-running the filter. This makes filters dramatically faster for things like term: {"status": "active"} or range: {"price": {"gte": 100}}. The best practice is: use must only for full-text queries where ranking matters; use filter for everything else (status, category, date range, etc.).
Key takeaways
- OpenSearch is an Apache 2.0 fork of Elasticsearch, powered by Lucene, accessed entirely via HTTP REST. The Docker stack runs it at
localhost:9200with security disabled for local development. - Field type is the most important mapping decision:
textfor full-text search (tokenised),keywordfor exact match and aggregations. Never aggregate on atextfield without a.keywordsub-field. - Set
number_of_replicas: 0andnumber_of_shards: 1for single-node local dev — avoids yellow cluster health and unnecessary overhead. - Use
filterclauses (notmust) for exact-match and range conditions — they are cached and orders of magnitude faster than scored queries for high-frequency filters.
Search, stream, and analyse at scale
Join the Best AI Program: The AI-Era Data Engineer — 5 months, fully online, with placement assistance. Week 1 is free.
Explore The AI-Era Data Engineer →