Home > Backend Development > Python Tutorial > How to Efficiently Remove Duplicate Dictionaries from a Python List?

How to Efficiently Remove Duplicate Dictionaries from a Python List?

Linda Hamilton
Release: 2024-11-01 11:29:02
Original
226 people have browsed it

How to Efficiently Remove Duplicate Dictionaries from a Python List?

Unique Dictionaries in Python Lists

Lists of dictionaries are common in Python applications. However, managing duplicate dictionaries can be challenging. This article discusses how to efficiently remove duplicates and obtain a list of unique dictionaries.

Consider a list of dictionaries:

<code class="python">L = [
    {'id': 1, 'name': 'john', 'age': 34},
    {'id': 1, 'name': 'john', 'age': 34},
    {'id': 2, 'name': 'hanna', 'age': 30}
]</code>
Copy after login

Approaching the Problem

To deduplicate a list of dictionaries, a straightforward approach involves iterating over the list and comparing each dictionary to the others. However, this process can be computationally expensive for large lists.

Using a Temporary Dictionary

A more efficient solution leverages a temporary dictionary to handle the deduplication. The key of the dictionary is set to the id field of each dictionary, and the value is set to the dictionary itself. This operation effectively filters out duplicates because each unique id will correspond to only one dictionary entry.

Retrieving Unique Dictionaries

Once the temporary dictionary is populated, the values (which represent the unique dictionaries) can be retrieved using the values() method.

Python Implementation

Python 2.7:

<code class="python">{v['id']:v for v in L}.values()</code>
Copy after login

Python 3:

<code class="python">list({v['id']:v for v in L}.values())</code>
Copy after login

Python 2.5/2.6:

<code class="python">dict((v['id'],v) for v in L).values()</code>
Copy after login

These concise solutions result in a list of unique dictionaries:

<code class="python">[
    {'id': 1, 'name': 'john', 'age': 34},
    {'id': 2, 'name': 'hanna', 'age': 30}
]</code>
Copy after login

This approach efficiently removes duplicates by leveraging a temporary dictionary to identify and extract unique dictionary values.

The above is the detailed content of How to Efficiently Remove Duplicate Dictionaries from a Python List?. For more information, please follow other related articles on the PHP Chinese website!

source:php.cn
Statement of this Website
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Latest Articles by Author
Popular Tutorials
More>
Latest Downloads
More>
Web Effects
Website Source Code
Website Materials
Front End Template