How to remove duplicates in python

How to remove duplicates in python? Here are several python methods to remove duplicates:

Method 1: Use the built-in set method to remove duplicates

>>> lst1 = [2, 1, 3, 4, 1]
>>> lst2 = list(set(lst1))
>>> print(lst2)
[1, 2, 3, 4]

Method 2: Use the fromkeys() method in the dictionary to remove duplicates

>>> lst1 = [2, 1, 3, 4, 1]
>>> lst2 = {}.fromkeys(lst1).keys()
>>> print(lst2)
dict_keys([2, 1, 3, 4])

Related recommendations: "python video tutorial"

Method 3: Use conventional methods to remove duplicates

>>> lst1 = [2, 1, 3, 4, 1]
>>> temp = []
>>> for item in lst1:
    if not item in temp:
    temp.append(item)
>>> print(temp)
[2, 1, 3, 4]

Method 4: Use list comprehension to remove duplicates

>>> lst1 = [2, 1, 3, 4, 1]
>>> temp = []
>>> [temp.append(i) for i in lst1 if not i in temp]
[None, None, None, None]
>>> print(temp)
[2, 1, 3, 4]

Method 5: Use the sort function to remove duplicates

>>> lst1 = [2, 1, 3, 4, 1]
>>> lst2.sort(key=lst1.index)
>>> print(lst2)
[2, 1, 3, 4]

Method 6: Use the sorted function to remove duplicates

>>> lst1 = [2, 1, 3, 4, 1]
>>> lst2 = sorted(set(lst1), key=lst1.index)
>>> print(lst2)
[2, 1, 3, 4]

Note: Some of the previous methods cannot guarantee their order. For example, use the set() function to handle it!

If you want to delete duplicate items in the list, you can also use the following methods to handle it

>>> # Method 1:

>>> data = [2, 1, 3, 4, 1]
>>> [item for item in data if data.count(item) == 1]

[2, 3, 4]

>>> # Method 2:

>>> data = [2, 1, 3, 4, 1]
>>> list(filter(lambda x:data.count(x) == 1, data))
[2, 3, 4]

The above is the detailed content of How to remove duplicates in python. For more information, please follow other related articles on the PHP Chinese website!