Union and intersection in Python - Deep Dive

Overview - Union and intersection

What is it?

Union and intersection are ways to combine or compare groups of items. Union means putting all items from two groups together without repeats. Intersection means finding only the items that appear in both groups. These concepts help organize and analyze collections of data.

Why it matters

Without union and intersection, it would be hard to merge or compare data sets efficiently. For example, combining friends lists or finding common interests would be slow and error-prone. These operations make data handling faster and clearer, which is important in many real-life tasks like searching, filtering, and organizing information.

Where it fits

Before learning union and intersection, you should understand basic data types like lists and sets in Python. After this, you can explore more complex set operations, like difference and symmetric difference, or apply these concepts in database queries and data science.

Mental Model

Core Idea

Union combines all unique items from two groups, while intersection finds only the items both groups share.

Think of it like...

Imagine two circles of friends at a party. Union is like inviting everyone from both circles to a new party, making sure no one is counted twice. Intersection is like finding the friends who are in both circles and inviting only them.

  Set A       Set B
  ┌─────┐    ┌─────┐
  │ 1 2 │    │ 2 3 │
  └─┬─┬─┘    └─┬─┬─┘
    │ │        │ │
    │ └───┐  ┌─┘ │
    │     │  │   │
    └─────┴──┴───┘

Union: {1, 2, 3}
Intersection: {2}

Build-Up - 7 Steps

1

Foundation

Understanding Python sets basics

Concept: Introduce sets as collections of unique items in Python.

In Python, a set is a group of unique items. You create a set using curly braces or the set() function. For example, set_a = {1, 2, 3} creates a set with three numbers. Sets automatically remove duplicates, so {1, 2, 2, 3} becomes {1, 2, 3}.

Result

Sets hold unique items only, no repeats.

Knowing that sets store unique items is key to understanding why union and intersection work the way they do.

2

Foundation

Creating and comparing sets

Concept: Learn how to create sets and check if items belong to them.

You can check if an item is in a set using the 'in' keyword, like 2 in set_a returns True. Sets are unordered, so their items have no fixed position. This means you can't access items by index but can quickly check membership.

Result

You can quickly test if items are in a set.

Understanding set membership helps you see why union and intersection are efficient for comparing groups.

3

Intermediate

Performing union with sets

🤔Before reading on: do you think union keeps duplicates or removes them? Commit to your answer.

Concept: Union combines all unique items from two sets into one set.

In Python, you can get the union of two sets using the | operator or the union() method. For example, set_a | set_b or set_a.union(set_b) returns a set with all unique items from both sets. Duplicates are removed automatically.

Result

Union returns a set with all unique items from both sets.

Knowing union removes duplicates helps you combine data without repeating items.

4

Intermediate

Finding intersection of sets

🤔Before reading on: do you think intersection returns items in either set or only those in both? Commit to your answer.

Concept: Intersection finds items that appear in both sets.

You can find the intersection using the & operator or the intersection() method. For example, set_a & set_b or set_a.intersection(set_b) returns a set with items common to both sets. Items not in both are excluded.

Result

Intersection returns only the items present in both sets.

Understanding intersection helps you find common elements quickly and clearly.

5

Intermediate

Using union and intersection with lists

Concept: Apply union and intersection concepts to lists by converting them to sets.

Lists can have duplicates and order, but sets do not. To find union or intersection of lists, convert them to sets first: set(list1) | set(list2) for union, set(list1) & set(list2) for intersection. Then convert back to list if needed.

Result

You can combine or compare lists using set operations.

Knowing how to convert lists to sets lets you use union and intersection on common data types.

6

Advanced

Performance benefits of set operations

🤔Before reading on: do you think union and intersection are faster or slower than looping through lists? Commit to your answer.

Concept: Set operations are optimized for speed compared to manual loops.

Sets use a special structure called a hash table that lets Python check membership and combine sets very fast. This makes union and intersection much quicker than checking items one by one in lists, especially for large data.

Result

Set operations run faster and use less code than manual loops.

Understanding the speed advantage explains why sets are preferred for these operations in real applications.

7

Expert

Subtle behavior with mutable and custom objects

🤔Before reading on: do you think sets can contain lists or custom objects by default? Commit to your answer.

Concept: Sets require items to be immutable and hashable; this affects union and intersection with complex data.

Sets cannot contain mutable types like lists because they can't be hashed. Custom objects can be in sets only if they implement special methods (__hash__ and __eq__). This means union and intersection work only with hashable items, which can surprise developers.

Result

Union and intersection fail or behave unexpectedly with unhashable items.

Knowing the hashability requirement prevents bugs when using sets with complex data types.

Under the Hood

Python sets are implemented using hash tables, which store items based on their hash value. When performing union or intersection, Python uses these hash values to quickly find unique items or common items without scanning every element. This makes these operations very efficient compared to lists.

Why designed this way?

Sets were designed to provide fast membership tests and set operations by leveraging hash tables. Alternatives like lists would require slower linear searches. Hash tables balance speed and memory use, making sets ideal for union and intersection tasks.

  ┌─────────────┐
  │   Set A     │
  │  hash table │
  │  {1, 2, 3}  │
  └─────┬───────┘
        │
        │ union/intersection
        ▼
  ┌─────────────┐
  │   Set B     │
  │  hash table │
  │  {2, 3, 4}  │
  └─────┬───────┘
        │
        ▼
  ┌─────────────────────┐
  │ Result Set (hash)    │
  │ Union: {1, 2, 3, 4} │
  │ Intersection: {2, 3} │
  └─────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does union keep duplicates from both sets or remove them? Commit to yes or no.

Common Belief: Union just combines all items, keeping duplicates from both sets.

Reality: Union removes duplicates and keeps only unique items from both sets.

Why it matters: Assuming duplicates remain can cause errors in counting or processing combined data.

Quick: Does intersection return items in either set or only those in both? Commit to your answer.

Common Belief: Intersection returns all items found in either set.

Reality: Intersection returns only items found in both sets simultaneously.

Why it matters: Misunderstanding this leads to wrong results when filtering common elements.

Quick: Can sets contain lists or other mutable objects? Commit to yes or no.

Common Belief: Sets can contain any type of item, including lists and dictionaries.

Reality: Sets can only contain hashable (immutable) items; lists and dicts are not allowed.

Why it matters: Trying to add mutable items causes runtime errors, confusing beginners.

Quick: Does the order of items matter in union and intersection results? Commit to yes or no.

Common Belief: The order of items in union and intersection matches the original sets.

Reality: Sets are unordered; the result order is arbitrary and should not be relied upon.

Why it matters: Expecting order can cause bugs when order matters, like displaying results.

Expert Zone

1

Union and intersection operations are optimized in C within Python, making them much faster than equivalent Python loops.

2

When chaining multiple unions or intersections, the order of operations can affect performance but not the final result.

3

Custom objects must implement __hash__ and __eq__ correctly to behave as expected in set operations, or subtle bugs can occur.

When NOT to use

Avoid sets when you need to preserve order or allow duplicates; use lists or specialized collections instead. For unhashable items, consider using lists with manual filtering or libraries like pandas for complex data.

Production Patterns

In real-world code, union and intersection are used for filtering user permissions, merging search results, and deduplicating data. They often appear in database query logic and data pipelines where fast set operations improve performance.

Connections

Database JOIN operations

Union and intersection correspond to SQL UNION and INNER JOIN operations.

Understanding set union and intersection helps grasp how databases combine or filter rows from tables.

Boolean logic

Union and intersection mirror OR and AND operations in Boolean algebra.

Knowing this connection clarifies how set operations relate to logical conditions and filtering.

Venn diagrams in mathematics

Union and intersection are visualized as overlapping areas in Venn diagrams.

Recognizing this helps understand the spatial relationship between sets and their combined or shared elements.

Common Pitfalls

#1 Trying to add a list to a set causes an error.

Wrong approach:

my_set = {1, 2, 3}
my_set.add([4, 5])

Correct approach:

my_set = {1, 2, 3}
my_set.update([4, 5])

Root cause:

Lists are mutable and unhashable, so they cannot be added as single items to sets; update adds each element instead.

#2 Expecting union to keep duplicates from both sets.

Wrong approach:

set_a = {1, 2}
set_b = {2, 3}
result = set_a.union(set_b)
print(result) # expecting {1, 2, 2, 3}

Correct approach:

set_a = {1, 2}
set_b = {2, 3}
result = set_a.union(set_b)
print(result) # outputs {1, 2, 3}

Root cause:

Sets automatically remove duplicates, so union never repeats items.

#3 Using lists directly for union without converting to sets.

Wrong approach:

list1 = [1, 2, 3]
list2 = [2, 3, 4]
result = list1 + list2
print(result) # outputs [1, 2, 3, 2, 3, 4]

Correct approach:

list1 = [1, 2, 3]
list2 = [2, 3, 4]
result = list(set(list1) | set(list2))
print(result) # outputs [1, 2, 3, 4]

Root cause:

Lists allow duplicates and concatenation just joins them; sets remove duplicates.

Key Takeaways

Union combines all unique items from two sets, removing duplicates automatically.

Intersection finds only the items that appear in both sets, filtering common elements.

Python sets require items to be immutable and hashable, which affects what can be included.

Set operations are much faster than manual loops for large data because of hash tables.

Understanding union and intersection helps in many areas like databases, logic, and data processing.