๐ฆ What is a Set?
A set is an unordered collection of unique items. Python enforces uniqueness automatically โ insert the same value twice and it stays once. Sets are fast for membership testing because they use hash tables underneath.
Two key traits: (1) no duplicates, and (2) checking if x in my_set takes O(1) time โ constant, not proportional to the set's size. This makes sets perfect for deduplication and fast lookups.
๐ Key Traits of Sets
๐ข Business Scenario: Email Deduplication at SouthBank India
SouthBank India is running a marketing campaign. They've collected customer emails from three sources: (1) their core database, (2) a partner bank, and (3) recent sign-ups. Some emails appear in multiple sources. They need to:
- Deduplicate the email list (remove repeats)
- Find which customers are in all three sources (high-value)
- Find customers only in their core database (to nurture)
๐ก Step 1: Deduplicating Email Lists with set()
Convert a list to a set to automatically remove duplicates. This is the simplest form of deduplication.
๐ฏ Step 2: Finding Common Customers (Intersection)
Use & or .intersection() to find emails appearing in multiple sources โ these are your most-engaged customers.
๐ Step 3: Planning the Campaign with Union & Difference
Use | (union) to combine lists, and - (difference) to exclude certain groups โ e.g., send to everyone except those who already engaged.
โก Step 4: Why Sets Are Fast โ O(1) vs. O(n)
The real power of sets: membership testing is constant-time. Test with timeit to see the speed difference when checking if an email is in a list vs. a set.
๐ Where Sets Show Up in Real Data Work
Common use cases
approved_merchants, flag it.patient_set1 & patient_set2 for overlap detection.action_movies & thriller_movies gives the intersection.๐ค How AI & ML Use Sets
Every recommendation engine and text classifier leans on sets. When a model learns "which words appear in spam emails", it builds a set of spam-words โ then checks new emails against that set in O(1) time. For large text corpora, this speed difference is enormous. In NLP feature engineering, you often use sets to find common_words_in_domain_1 & common_words_in_domain_2 โ words that define both genres of text.
๐ฏ Mini Challenge
You have two user segment tags from Jumia Kenya. segment_a = {"electronics", "premium", "urban"} and segment_b = {"premium", "mobile-first", "deals"}. Find: (1) tags in both segments (intersection), (2) tags unique to segment_a (difference), (3) all tags combined (union). Then find what you'd exclude: tags in segment_b that are not in segment_a.
& for intersection, - for difference, and | for union. Remember sets are unordered, so print them or convert to sorted lists for consistent output.
๐ค Interview Questions
[1, 2, 2, 3] has four items. A set is unordered and enforces uniqueness: {1, 2, 3} has three items. Lists support indexing (list[0]); sets do not. Use lists when order matters or duplicates are intentional; use sets for fast membership testing and deduplication.if x in list is O(n), meaning it scales with list length. For large datasets, this difference is dramatic: searching a list of 1 million items takes ~500,000 comparisons on average; a set takes one hash calculation.remove() raises KeyError if the item doesn't exist. discard() silently does nothing if the item is missing. In production code, use discard() when you're not certain the item is in the set โ it won't crash your pipeline. Use remove() when the item must be there, and you want to catch the error immediately.tuple is immutable, so it's hashable. list is mutable; you can change its contents after creation, which would break the hash table. If you want to store a group of items in a set, wrap them in a tuple: set_of_combos = {(1, 2), (3, 4)}. This is crucial when storing database keys or multi-field records.frozenset is immutable โ you can't add or remove items after creation. Use it when you need a set as a dictionary key or want to ensure it doesn't change. Example: if you're building a cache of common email sets, use frozenset keys so they can be hashed in a dict. Regular sets are unhashable, so {set1, set2} fails, but {frozenset1, frozenset2} works. Also use frozenset when passing sets between functions and you don't want them modified.๐ง Quick Quiz
set([1, 2, 2, 3]) return?x in my_set?remove() on a missing item?๐ Quick Cheatsheet
| Task | Syntax | Notes |
|---|---|---|
| Create a set | s = {1, 2, 3} or set(list) | Curly braces, not square brackets |
| Add item | s.add(4) | Silently ignores if item exists |
| Remove item (safe) | s.discard(4) | No error if missing |
| Remove item (strict) | s.remove(4) | KeyError if missing |
| Intersection | s1 & s2 or s1.intersection(s2) | Items in both |
| Union | s1 | s2 or s1.union(s2) | Items in either |
| Difference | s1 - s2 or s1.difference(s2) | Items in s1 but not s2 |
| Membership test | if x in s: | O(1) โ constant time |
| Frozenset | fs = frozenset([1, 2]) | Immutable; can be dict key |
โ ๏ธ Common Mistakes
{} creates an empty dict, not a set. Use set() for an empty set. Confusing the two can silently break your code.my_set[0] raises TypeError. If you need the first item, convert to a list: list(my_set)[0]. But remember the order is unpredictable.{[1, 2], [3, 4]} raises TypeError: unhashable type. Use tuples instead: {(1, 2), (3, 4)}. Tuples are immutable and hashable.set([1, 2, 3]) will iterate in that order. If order matters, convert to a sorted list: sorted(my_set). Sets are designed for membership testing, not ordering.โฆ Key Takeaways
- Sets automatically enforce uniqueness โ duplicate values are silently dropped
- Membership testing on sets is O(1) โ orders of magnitude faster than lists for large data
- Use
&for intersection,|for union,-for difference โ like Venn diagrams - Set elements must be hashable (immutable): str, int, tuple yes; list, dict, set no
discard()is safer thanremove()in production code when item may be missingfrozensetis immutable and can be used as a dict key or in another set
You've learnt sets.
Now master data structures and deduplication.
Join the Best AI Program: The AI-Era Analyst โ 5 months, fully online, with placement assistance. Week 1 is free.
Explore The AI-Era Analyst โ