The Pipeline Concept

The aggregation pipeline is MongoDB's powerhouse for analytics. Instead of retrieving all data and processing it in your application, the pipeline transforms data inside MongoDB using stages that flow in sequence.

Think of it like a factory assembly line: raw documents enter Stage 1, get transformed, move to Stage 2, get transformed again, and so on. By the time data exits the pipeline, you have exactly what you need.

Pipeline Flow: $match → filters documents early (reduces data processed) → $group → aggregates by key → $sort → orders results → $project → selects final fields.

The key insight: $match early. Filtering documents at the start of the pipeline is much faster than grouping everything and then filtering.

Why Aggregation Matters

  • Efficiency: Processing happens on the database server, not your application.
  • Complex Queries: Group by multiple fields, compute running totals, reshape nested arrays.
  • Analytics: Revenue per category, top customers, supplier performance, all in one query.

Grouping & Accumulators

The $group stage is where the magic happens. It groups documents by a field and accumulates values using operators like $sum, $avg, $min, $max, $push, $first, and $last.

Q1: Count Products by Category

Let's count products in each category and calculate average price:

javascript
Sample: { _id: "Vegetables", count: 10, avg_price: 1229.9 }

Breakdown:

  • _id: "$category" — Groups by the category field.
  • $sum: 1 — Counts documents (increment by 1 for each).
  • $avg: "$price" — Averages the price field.

Q2: Group with Multiple Accumulators

Use $min and $max to find the price range per category:

javascript
Sample: { _id: "Vegetables", min_price: 163, max_price: 2307, total_stock: 1447 }

All Accumulators:

  • $sum — Total or count.
  • $avg — Average value.
  • $min / $max — Smallest / largest value.
  • $push — Collect all values into an array.
  • $first / $last — First or last document in group.

Q3: Collect Product Names into Array

Use $push to create arrays of product names per category:

javascript
Sample: { _id: "Vegetables", product_names: ["Tomatoes 1kg", "Onions 2kg", ...] }

$unwind — Flattening Arrays

In MongoDB, documents can have nested arrays. The $unwind stage flattens an array field, creating one document per array element. This is perfect for analyzing items in orders.

Understanding $unwind

Imagine an order document with multiple items. $unwind creates a separate document for each item, allowing you to aggregate by product.

javascript
After $unwind:

Q4: Total Units Sold Per Product

Unwind orders, then group by product_id to sum quantities:

javascript
Sample: { _id: "prod_72", total_sold: 8 }, { _id: "prod_36", total_sold: 20 }

Step-by-step:

  • $unwind flattens the items array → each item becomes its own document.
  • $group by product_id → sums quantities across all orders.

Q5: Revenue Per Product

Calculate total revenue (quantity × unit_price) per product, top 10:

javascript
Sample: { _id: "prod_17", product_name: "Cabbage 1pc", units_sold: 30, total_revenue: 55080 }

$lookup — JOINs in MongoDB

MongoDB doesn't have traditional SQL JOINs, but $lookup does the same thing: it links data from two collections.

The $lookup Syntax

javascript

Q6: Join Orders with Customer Details

For each order, attach the matching customer information:

javascript
Each order now has customer_details array with matching customer document.

Q7: Full Pipeline — Unwind, Lookup, Group

Advanced example: Join orders with customers, unwind items, group by supplier, calculate top revenue:

javascript

$addFields, $project, $bucket

$addFields — Computed Fields

$addFields adds new fields to documents using expressions. Calculate inventory value (price × stock):

javascript
Sample: { name: "Birthday Cake 1kg", price: 2443, stock: 276, inventory_value: 674268 }

$project — Reshaping Documents

$project selects, renames, and transforms fields. Include only customer_id, total, and computed fields:

javascript
Sample: { customer_id: "cust_15", total: 19237, item_count: 5, status_upper: "DELIVERED" }

Note: 0 = exclude field, 1 = include field. You can compute fields with operators like $size, $toUpper, $multiply, etc.

$bucket — Grouping by Price Ranges

$bucket groups documents into price ranges. No product is left unbucketed:

javascript
Sample: { _id: 0, count: 9, products: ["Papaya 1pc", "Grapes 500g", ...] }
  • boundaries — Define bucket ranges: [0-200), [200-400), [400-600), etc.
  • default — Catch-all bucket for values outside boundaries.
  • output — Define what to include in each bucket.

Complex Pipelines

Q8: Supplier Performance Report

Group by supplier, calculate product count, average rating, and average price. Sort by rating (top first):

javascript
Sample: { _id: "Ratnagiri Farms", supplier_location: "India", product_count: 6, avg_rating: 4.28, avg_price: 1432.5 }

Q9: Multi-Stage Order Insights

Find top revenue-generating products from delivered orders only:

javascript
Sample: { _id: "prod_17", product_name: "Cabbage 1pc", total_revenue: 45900, orders_count: 9, units_sold: 25 }

Q10: Day-over-Day Orders & Revenue

Group orders by delivery date, calculate daily order count and revenue:

javascript
Sample: { _id: "2025-06-27", order_count: 1, revenue: 17109 }

Challenge

Write an aggregation pipeline to find the top 5 customers by lifetime spend. Include:

  • Customer name and city.
  • Total orders placed.
  • Lifetime spend (sum of all order totals).
  • Average spend per order (lifetime_spend / order_count).
  • Sort by lifetime spend descending.
Show hint

Here's a template to follow:

javascript

Test Your Knowledge

Question 1 of 5
What does $unwind do?
Question 2 of 5
How do you join collections in MongoDB?
Question 3 of 5
Which three are valid accumulator operators in $group?
Question 4 of 5
What is the purpose of $bucket?
Question 5 of 5
Should $match come before or after $group?
0
/5
Africa & India's Best AI Academy
🤖 AI-first 🎓 Placement Assistance

Aggregation pipelines — MongoDB's superpower.

Join the Best AI Program: The AI-Era Data Engineer — 5 months, fully online, with placement assistance. Week 1 is free.

Explore The AI-Era Data Engineer →