Remove Duplicates in One Line Python: Simple and Fast

You can remove duplicates in one line in Python by converting a list to a set using list(set(your_list)), which removes duplicates but does not keep order. To keep order, use list(dict.fromkeys(your_list)) which removes duplicates while preserving the original order.

Syntax

There are two common one-line ways to remove duplicates from a list in Python:

• list(set(your_list)): Converts the list to a set to remove duplicates, then back to a list. Order is not preserved.
• list(dict.fromkeys(your_list)): Uses a dictionary to remove duplicates while keeping the original order.

python

unique_list = list(set(your_list))

# or to keep order
unique_list_ordered = list(dict.fromkeys(your_list))

Example

This example shows how to remove duplicates from a list of numbers using both methods and prints the results.

python

your_list = [3, 1, 2, 3, 4, 2, 1]

# Remove duplicates without preserving order
unique_no_order = list(set(your_list))

# Remove duplicates and preserve order
unique_with_order = list(dict.fromkeys(your_list))

print("Without order:", unique_no_order)
print("With order:", unique_with_order)

Output

Without order: [1, 2, 3, 4]
With order: [3, 1, 2, 4]

Common Pitfalls

Using list(set(your_list)) removes duplicates but does not keep the original order, which can be confusing if order matters. Also, sets only work with hashable items, so this method fails with unhashable types like lists inside the list.

Using dict.fromkeys() preserves order and works with hashable items, but if your list contains unhashable types, it will raise an error.

python

your_list = [1, 2, [3, 4], 2]

# This will raise an error because list is unhashable
# unique = list(set(your_list))  # TypeError

# dict.fromkeys also fails with unhashable types
# unique_ordered = list(dict.fromkeys(your_list))  # TypeError

# Correct approach: use a loop or other methods for unhashable items

Quick Reference

Summary of one-line duplicate removal methods:

Method	Removes Duplicates	Preserves Order	Notes
list(set(your_list))	Yes	No	Fast, order not preserved, only hashable items
list(dict.fromkeys(your_list))	Yes	Yes	Preserves order, only hashable items

Key Takeaways

Use list(set(your_list)) to quickly remove duplicates without keeping order.

Use list(dict.fromkeys(your_list)) to remove duplicates and keep the original order.

Both methods require list items to be hashable types like numbers or strings.

If your list contains unhashable items, these one-line methods will raise errors.

Choose the method based on whether you need to preserve the order of elements.