Python list deduplication

Python list list deduplication

一.{}.fromkeys(list).keys()

list2 = {}.fromkeys(list1).keys()

二.set

list2 = list (set(list1))

三.itertools.grouby

ids = [1,4,3,3,4,2,3,4,5,6,1]

ids.sort()

it = itertools.groupby(ids)

for k, g in it:

print k

Four, stupid method

ids = [1,2,3,3,4,2,3,4 ,5,6,1]

news_ids = []

for id in ids:

if id not in news_ids:

news_ids.append(id)

print news_ids

These four have their own characteristics, After deduplication, the sorting of elements has changed. It is said that the first method is faster than the second one

5. Index sorting again can remove duplication and maintain the order of elements

#The result is [1, 4, 3, 2, 5, 6] Don’t [1, 2, 3, 4, 5, 6]

ids = [1,4,3,3,4,2,3,4,5,6,1]

news_ids = list(set(ids))

news_ids.sort(key=ids.index)

print news_ids #[1, 4, 3, 2, 5, 6]

Six: Reduce

ids = [ 1,4,3,3,4,2,3,4,5,6,1]

func = lambda x,y:x if y in x else x + [y]

print reduce(func, [ [], ] + ids)#[1, 4, 3, 2, 5, 6]