Python Program to Remove Duplicates from List

You can remove duplicates from a list in Python by converting it to a set and back to a list using list(set(your_list)), or by using a loop to keep only unique items.

Examples

Input[1, 2, 2, 3, 4, 4, 5]

Output[1, 2, 3, 4, 5]

Input['apple', 'banana', 'apple', 'orange']

Output['apple', 'banana', 'orange']

Input[]

Output[]

How to Think About It

To remove duplicates, think about how to keep only one copy of each item. One easy way is to use a set, which automatically removes duplicates because it only stores unique values. Then, convert the set back to a list if you want a list result. Another way is to check each item and add it to a new list only if it is not already there.

Algorithm

1

Get the input list.

2

Create an empty list to store unique items.

3

For each item in the input list, check if it is not already in the unique list.

4

If not present, add the item to the unique list.

5

Return or print the unique list.

Code

python

def remove_duplicates(lst):
    unique_list = []
    for item in lst:
        if item not in unique_list:
            unique_list.append(item)
    return unique_list

# Example usage
my_list = [1, 2, 2, 3, 4, 4, 5]
print(remove_duplicates(my_list))

Output

[1, 2, 3, 4, 5]

Dry Run

Let's trace the list [1, 2, 2, 3, 4, 4, 5] through the code

1

Start with empty unique_list

unique_list = []

2

Check first item 1

1 not in unique_list, so add 1 -> unique_list = [1]

3

Check second item 2

2 not in unique_list, add 2 -> unique_list = [1, 2]

4

Check third item 2

2 already in unique_list, skip

5

Check fourth item 3

3 not in unique_list, add 3 -> unique_list = [1, 2, 3]

6

Check fifth item 4

4 not in unique_list, add 4 -> unique_list = [1, 2, 3, 4]

7

Check sixth item 4

4 already in unique_list, skip

8

Check seventh item 5

5 not in unique_list, add 5 -> unique_list = [1, 2, 3, 4, 5]

Iteration	Current Item	unique_list
1	1	[1]
2	2	[1, 2]
3	2	[1, 2]
4	3	[1, 2, 3]
5	4	[1, 2, 3, 4]
6	4	[1, 2, 3, 4]
7	5	[1, 2, 3, 4, 5]

Why This Works

Step 1: Using a loop to check duplicates

The code goes through each item and checks if it is already in the new list using if item not in unique_list. This ensures only unique items are added.

Step 2: Building a new list

Instead of changing the original list, the code creates a new list to store unique items, which keeps the order of first appearances.

Step 3: Returning the result

After checking all items, the new list with no duplicates is returned and printed.

Alternative Approaches

Using set conversion

python

def remove_duplicates_set(lst):
    return list(set(lst))

print(remove_duplicates_set([1, 2, 2, 3, 4, 4, 5]))

This is the fastest way but does not keep the original order of items.

Using dict.fromkeys()

python

def remove_duplicates_dict(lst):
    return list(dict.fromkeys(lst))

print(remove_duplicates_dict([1, 2, 2, 3, 4, 4, 5]))

This keeps the order and is concise, using Python 3.7+ where dict preserves insertion order.

Complexity: O(n^2) time, O(n) space

Time Complexity

The main method uses a loop with if item not in unique_list, which takes O(n) time for each check, leading to O(n^2) overall for n items.

Space Complexity

It uses extra space for the new list to store unique items, so O(n) space is needed.

Which Approach is Fastest?

Using set() conversion is fastest with O(n) time but loses order. Using dict.fromkeys() is also O(n) and keeps order, making it a good balance.

Approach	Time	Space	Best For
Loop with list check	O(n^2)	O(n)	Small lists, order preserved
Set conversion	O(n)	O(n)	Fastest, order not preserved
dict.fromkeys()	O(n)	O(n)	Fast and order preserved