Python Program to Remove Duplicate Items from a List of Dictionaries

by

Here is a Python program to remove duplicate items from a list of dictionaries while keeping the order.

Remove Duplicate Items From a Dictionary in Python and Keep The Order

The following Python program will remove the two duplicate records for {'APPLE': 20, 'MANGO': 30} entries from the list of directories.

def dedupe(items, key=None):
    seen = set()
    for item in items:
        val = item if key is None else key(item)
        if val not in seen:
            yield item
            seen.add(val)

if __name__ == '__main__':
    a = [
        {'APPLE': 20, 'MANGO': 30},
        {'APPLE': 10, 'MANGO': 40},
        {'APPLE': 20, 'MANGO': 30},
        {'APPLE': 20, 'MANGO': 30},
        {'APPLE': 10, 'MANGO': 15}
        ]
#print(a)
    print(list(dedupe(a, key=lambda a: (a['APPLE'],a['MANGO']))))

Output:

[{'APPLE': 20, 'MANGO': 30}, {'APPLE': 10, 'MANGO': 40}, {'APPLE': 10, 'MANGO': 15}]

Process finished with exit code 0

See also:

Leave a Comment