Union and intersection in Python - Deep Dive

Overview - Union and intersection

What is it?

Union and intersection are ways to combine or compare groups of items. Union means putting all items from two groups together without repeats. Intersection means finding only the items that appear in both groups. These concepts help organize and analyze collections of data.

Why it matters

Without union and intersection, it would be hard to merge or compare data sets efficiently. For example, combining friends lists or finding common interests would be slow and error-prone. These operations make data handling faster and clearer, which is important in many real-life tasks like searching, filtering, and organizing information.

Where it fits

Before learning union and intersection, you should understand basic data types like lists and sets in Python. After this, you can explore more complex set operations, like difference and symmetric difference, or apply these concepts in database queries and data science.

Mental Model

Core Idea

Union combines all unique items from two groups, while intersection finds only the items both groups share.

Think of it like...

Imagine two circles of friends at a party. Union is like inviting everyone from both circles to a new party, making sure no one is counted twice. Intersection is like finding the friends who are in both circles and inviting only them.

  Set A       Set B
  ┌─────┐    ┌─────┐
  │ 1 2 │    │ 2 3 │
  └─┬─┬─┘    └─┬─┬─┘
    │ │        │ │
    │ └───┐  ┌─┘ │
    │     │  │   │
    └─────┴──┴───┘

Union: {1, 2, 3}
Intersection: {2}

Build-Up - 7 Steps

1

Foundation

Understanding Python sets basics

Concept: Introduce sets as collections of unique items in Python.

In Python, a set is a group of unique items. You create a set using curly braces or the set() function. For example, set_a = {1, 2, 3} creates a set with three numbers. Sets automatically remove duplicates, so {1, 2, 2, 3} becomes {1, 2, 3}.

Result

Sets hold unique items only, no repeats.

Knowing that sets store unique items is key to understanding why union and intersection work the way they do.

2

Foundation

Creating and comparing sets

Concept: Learn how to create sets and check if items belong to them.

You can check if an item is in a set using the 'in' keyword, like 2 in set_a returns True. Sets are unordered, so their items have no fixed position. This means you can't access items by index but can quickly check membership.

Result

You can quickly test if items are in a set.

Understanding set membership helps you see why union and intersection are efficient for comparing groups.

3

Intermediate

Performing union with sets

🤔Before reading on: do you think union keeps duplicates or removes them? Commit to your answer.

Concept: Union combines all unique items from two sets into one set.

In Python, you can get the union of two sets using the | operator or the union() method. For example, set_a | set_b or set_a.union(set_b) returns a set with all unique items from both sets. Duplicates are removed automatically.

Result

Union returns a set with all unique items from both sets.

Knowing union removes duplicates helps you combine data without repeating items.

4

Intermediate

Finding intersection of sets

🤔Before reading on: do you think intersection returns items in either set or only those in both? Commit to your answer.

Concept: Intersection finds items that appear in both sets.

You can find the intersection using the & operator or the intersection() method. For example, set_a & set_b or set_a.intersection(set_b) returns a set with items common to both sets. Items not in both are excluded.

Result

Intersection returns only the items present in both sets.

Understanding intersection helps you find common elements quickly and clearly.

5

Intermediate

Using union and intersection with lists

Concept: Apply union and intersection concepts to lists by converting them to sets.

Lists can have duplicates and order, but sets do not. To find union or intersection of lists, convert them to sets first: set(list1) | set(list2) for union, set(list1) & set(list2) for intersection. Then convert back to list if needed.

Result

You can combine or compare lists using set operations.

Knowing how to convert lists to sets lets you use union and intersection on common data types.

6

Advanced

Performance benefits of set operations

🤔Before reading on: do you think union and intersection are faster or slower than looping through lists? Commit to your answer.

Concept: Set operations are optimized for speed compared to manual loops.

Sets use a special structure called a hash table that lets Python check membership and combine sets very fast. This makes union and intersection much quicker than checking items one by one in lists, especially for large data.

Result

Set operations run faster and use less code than manual loops.

Understanding the speed advantage explains why sets are preferred for these operations in real applications.

7

Expert

Subtle behavior with mutable and custom objects

🤔Before reading on: do you think sets can contain lists or custom objects by default? Commit to your answer.

Concept: Sets require items to be immutable and hashable; this affects union and intersection with complex data.

Sets cannot contain mutable types like lists because they can't be hashed. Custom objects can be in sets only if they implement special methods (__hash__ and __eq__). This means union and intersection work only with hashable items, which can surprise developers.

Result

Union and intersection fail or behave unexpectedly with unhashable items.

Knowing the hashability requirement prevents bugs when using sets with complex data types.

Under the Hood

Python sets are implemented using hash tables, which store items based on their hash value. When performing union or intersection, Python uses these hash values to quickly find unique items or common items without scanning every element. This makes these operations very efficient compared to lists.

Why designed this way?

Sets were designed to provide fast membership tests and set operations by leveraging hash tables. Alternatives like lists would require slower linear searches. Hash tables balance speed and memory use, making sets ideal for union and intersection tasks.

  ┌─────────────┐
  │   Set A     │
  │  hash table │
  │  {1, 2, 3}  │
  └─────┬───────┘
        │
        │ union/intersection
        ▼
  ┌─────────────┐
  │   Set B     │
  │  hash table │
  │  {2, 3, 4}  │
  └─────┬───────┘
        │
        ▼
  ┌─────────────────────┐
  │ Result Set (hash)    │
  │ Union: {1, 2, 3, 4} │
  │ Intersection: {2, 3} │
  └─────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does union keep duplicates from both sets or remove them? Commit to yes or no.

Common Belief: Union just combines all items, keeping duplicates from both sets.

Reality: Union removes duplicates and keeps only unique items from both sets.

Why it matters: Assuming duplicates remain can cause errors in counting or processing combined data.

Quick: Does intersection return items in either set or only those in both? Commit to your answer.

Common Belief: Intersection returns all items found in either set.

Reality: Intersection returns only items found in both sets simultaneously.

Why it matters: Misunderstanding this leads to wrong results when filtering common elements.

Quick: Can sets contain lists or other mutable objects? Commit to yes or no.

Common Belief: Sets can contain any type of item, including lists and dictionaries.

Reality: Sets can only contain hashable (immutable) items; lists and dicts are not allowed.

Why it matters: Trying to add mutable items causes runtime errors, confusing beginners.

Quick: Does the order of items matter in union and intersection results? Commit to yes or no.

Common Belief: The order of items in union and intersection matches the original sets.

Reality: Sets are unordered; the result order is arbitrary and should not be relied upon.

Why it matters: Expecting order can cause bugs when order matters, like displaying results.

Expert Zone

1

Union and intersection operations are optimized in C within Python, making them much faster than equivalent Python loops.

2

When chaining multiple unions or intersections, the order of operations can affect performance but not the final result.

3

Custom objects must implement __hash__ and __eq__ correctly to behave as expected in set operations, or subtle bugs can occur.

When NOT to use

Avoid sets when you need to preserve order or allow duplicates; use lists or specialized collections instead. For unhashable items, consider using lists with manual filtering or libraries like pandas for complex data.

Production Patterns

In real-world code, union and intersection are used for filtering user permissions, merging search results, and deduplicating data. They often appear in database query logic and data pipelines where fast set operations improve performance.

Connections

Database JOIN operations

Union and intersection correspond to SQL UNION and INNER JOIN operations.

Understanding set union and intersection helps grasp how databases combine or filter rows from tables.

Boolean logic

Union and intersection mirror OR and AND operations in Boolean algebra.

Knowing this connection clarifies how set operations relate to logical conditions and filtering.

Venn diagrams in mathematics

Union and intersection are visualized as overlapping areas in Venn diagrams.

Recognizing this helps understand the spatial relationship between sets and their combined or shared elements.

Common Pitfalls

#1 Trying to add a list to a set causes an error.

Wrong approach:

my_set = {1, 2, 3}
my_set.add([4, 5])

Correct approach:

my_set = {1, 2, 3}
my_set.update([4, 5])

Root cause:

Lists are mutable and unhashable, so they cannot be added as single items to sets; update adds each element instead.

#2 Expecting union to keep duplicates from both sets.

Wrong approach:

set_a = {1, 2}
set_b = {2, 3}
result = set_a.union(set_b)
print(result) # expecting {1, 2, 2, 3}

Correct approach:

set_a = {1, 2}
set_b = {2, 3}
result = set_a.union(set_b)
print(result) # outputs {1, 2, 3}

Root cause:

Sets automatically remove duplicates, so union never repeats items.

#3 Using lists directly for union without converting to sets.

Wrong approach:

list1 = [1, 2, 3]
list2 = [2, 3, 4]
result = list1 + list2
print(result) # outputs [1, 2, 3, 2, 3, 4]

Correct approach:

list1 = [1, 2, 3]
list2 = [2, 3, 4]
result = list(set(list1) | set(list2))
print(result) # outputs [1, 2, 3, 4]

Root cause:

Lists allow duplicates and concatenation just joins them; sets remove duplicates.

Key Takeaways

Union combines all unique items from two sets, removing duplicates automatically.

Intersection finds only the items that appear in both sets, filtering common elements.

Python sets require items to be immutable and hashable, which affects what can be included.

Set operations are much faster than manual loops for large data because of hash tables.

Understanding union and intersection helps in many areas like databases, logic, and data processing.

Practice

1. What does the union operation do when applied to two sets in Python?

easy

A. Combines all unique elements from both sets

B. Finds elements only in the first set

C. Finds elements only in the second set

D. Finds elements common to both sets

Solution

1. Step 1: Understand union operation

   The union of two sets includes every unique element from both sets without duplicates.

2. Step 2: Compare with other options

   Options B and C describe subsets, and D describes intersection, not union.

3. Final Answer:

   Combines all unique elements from both sets -> Option A

4. Quick Check:

   Union = all unique elements combined [OK]

Hint: Union joins all unique items from both sets [OK]

Common Mistakes:

• Confusing union with intersection
• Thinking union only takes elements from one set
• Assuming union keeps duplicates

2. Which of the following is the correct syntax to find the intersection of two sets a and b in Python?

easy

A. a + b

B. a | b

C. a & b

D. a - b

Solution

1. Step 1: Recall intersection syntax

   In Python, the intersection of two sets is found using the & operator or the .intersection() method.

2. Step 2: Check other operators

   | is union, + is invalid for sets, - is difference.

3. Final Answer:

   a & b -> Option C

4. Quick Check:

   Intersection uses & operator [OK]

Hint: Use & for intersection of sets [OK]

Common Mistakes:

• Using | instead of & for intersection
• Trying to add sets with + operator
• Confusing difference (-) with intersection

3. What is the output of the following code?

set1 = {1, 2, 3, 4}
set2 = {3, 4, 5, 6}
print(set1.intersection(set2))

medium

A. {1, 2}

B. {5, 6}

C. {1, 2, 3, 4, 5, 6}

D. {3, 4}

Solution

1. Step 1: Identify intersection elements

   The intersection contains elements present in both sets: {3, 4}.

2. Step 2: Confirm output

   Printing the intersection returns {3, 4} exactly.

3. Final Answer:

   {3, 4} -> Option D

4. Quick Check:

   Common elements = {3, 4} [OK]

Hint: Intersection shows only common elements [OK]

Common Mistakes:

• Confusing union with intersection output
• Expecting all elements combined
• Mixing up set contents

4. The following code is intended to print the union of two sets, but it causes an error. What is the problem?

set1 = {1, 2, 3}
set2 = {3, 4, 5}
print(set1 + set2)

medium

A. Sets cannot be added with + operator

B. The sets are empty

C. The print statement is missing parentheses

D. The sets have overlapping elements

Solution

1. Step 1: Identify the error cause

   Python sets do not support the + operator; this causes a TypeError.

2. Step 2: Correct way to union sets

   Use | operator or .union() method to combine sets.

3. Final Answer:

   Sets cannot be added with + operator -> Option A

4. Quick Check:

   + operator invalid for sets [OK]

Hint: Use | or .union(), not + for sets [OK]

Common Mistakes:

• Trying to add sets with +
• Ignoring error message
• Confusing list addition with set union

5. Given two lists of student names, list1 = ['Anna', 'Bob', 'Cara'] and list2 = ['Bob', 'Diana', 'Anna'], which Python code correctly finds students present in both lists using set operations?

hard

A. list1 | list2

B. set(list1) & set(list2)

C. set(list1) + set(list2)

D. list1.intersection(list2)

Solution

1. Step 1: Convert lists to sets for set operations

   Lists must be converted to sets to use intersection (&) operator.

2. Step 2: Use & operator to find common elements

   set(list1) & set(list2) returns elements in both sets.

3. Step 3: Check other options

   list1 | list2 uses | on lists (invalid), C uses + on sets (invalid), D calls intersection on list (no such method).

4. Final Answer:

   set(list1) & set(list2) -> Option B

5. Quick Check:

   Convert lists to sets, then & for intersection [OK]

Hint: Convert lists to sets before intersection [OK]

Common Mistakes:

• Using + or | on lists directly
• Calling set methods on lists
• Not converting lists to sets first