How to Remove Duplicates in List in Python

Using Built-in set Function

The built-in set() function returns a new set object, optionally with elements taken from iterable. If you later need a real list again, you can similarly pass the set to the list() function.

The following example should cover whatever you are trying to do:

#!/usr/bin/python3

a = [1, 3, 1, 3, 2]

b = list(set(a))
print(b)

[1, 2, 3]

As you can see from the example result, the original order is not maintained. As mentioned above, sets themselves are unordered collections, so the order is lost.

Using OrderedDict.fromkeys Method

Return an instance of a dict subclass that has methods specialized for rearranging dictionary order.

If order is important to you, then you will have to use a different mechanism. A very common solution for this is to rely on OrderedDict to keep the order of keys during insertion:

#!/usr/bin/python3

# Import module
from collections import OrderedDict

a = [1, 3, 1, 3, 2]
b = list(OrderedDict.fromkeys(a))
print(b)

[1, 3, 2]

Using dict.fromkeys Method

Starting with Python 3.7, the built-in dictionary is guaranteed to maintain the insertion order as well:

#!/usr/bin/python3

a = [1, 3, 1, 3, 2]
# Python 3.7+
b = list(dict.fromkeys(a))
print(b)

[1, 3, 2]