Given a list of dictionaries, remove all the dictionaries which are duplicate with respect to K key.
Input : test_list = [{“Gfg” : 6, “is” : 9, “best” : 10}, {“Gfg” : 8, “is” : 11, “best” : 10}, {“Gfg” : 2, “is” : 16, “best” : 10}], K = “best” Output : [{“Gfg” : 6, “is” : 9, “best” : 10}] Explanation : All keys have 10 value, only 1st record is retained. Input : test_list = [{“Gfg” : 6, “is” : 9, “best” : 10}, {“Gfg” : 8, “is” : 11, “best” : 12}, {“Gfg” : 2, “is” : 16, “best” : 15}], K = “best” Output : [{“Gfg” : 6, “is” : 9, “best” : 10}, {“Gfg” : 8, “is” : 11, “best” : 12}, {“Gfg” : 2, “is” : 16, “best” : 15}] Explanation : All values of “best” are unique, hence no removal of dictionaries.
Method : Using loop
This is brute way in which this task can be performed. In this, we iterate for each dictionary and memoize the Key, if similar key’s same value occur, then that dictionary is avoided in resultant list of dictionaries.
Python3
# Python3 code to demonstrate working of # Remove Duplicate Dictionaries characterized by Key# Using loop# initializing liststest_list = [{"Gfg" : 6, "is" : 9, "best" : 10}, {"Gfg" : 8, "is" : 11, "best" : 19}, {"Gfg" : 2, "is" : 16, "best" : 10}, {"Gfg" : 12, "is" : 1, "best" : 8}, {"Gfg" : 22, "is" : 6, "best" : 8}]# printing original listprint("The original list : " + str(test_list))# initializing Key K = "best"memo = set()res = []for sub in test_list: # testing for already present value if sub[K] not in memo: res.append(sub) # adding in memo if new value memo.add(sub[K]) # printing result print("The filtered list : " + str(res)) |
The original list : [{‘Gfg’: 6, ‘is’: 9, ‘best’: 10}, {‘Gfg’: 8, ‘is’: 11, ‘best’: 19}, {‘Gfg’: 2, ‘is’: 16, ‘best’: 10}, {‘Gfg’: 12, ‘is’: 1, ‘best’: 8}, {‘Gfg’: 22, ‘is’: 6, ‘best’: 8}] The filtered list : [{‘Gfg’: 6, ‘is’: 9, ‘best’: 10}, {‘Gfg’: 8, ‘is’: 11, ‘best’: 19}, {‘Gfg’: 12, ‘is’: 1, ‘best’: 8}]
METHOD 2 : use list comprehension.
Here are the steps to implement it:
- Initialize the input list:
- Specify the key to be used for identifying duplicates
- Use list comprehension to filter out duplicates:
- This line of code creates a set of tuples where each tuple represents a dictionary in the input list. The dict() function is then used to convert each tuple back into a dictionary, and the resulting list contains only unique dictionaries.
- Print the filtered list:
Python3
# Initializing the input listtest_list = [{"Gfg" : 6, "is" : 9, "best" : 10}, {"Gfg" : 8, "is" : 11, "best" : 19}, {"Gfg" : 2, "is" : 16, "best" : 10}, {"Gfg" : 12, "is" : 1, "best" : 8}, {"Gfg" : 22, "is" : 6, "best" : 8}]# Specify the key to be used for identifying duplicatesK = "best"# Use list comprehension to filter out duplicatesres = [dict(t) for t in {tuple(d.items()) for d in test_list}]# Print the filtered listprint("The filtered list : " + str(res)) |
The filtered list : [{'Gfg': 2, 'is': 16, 'best': 10}, {'Gfg': 8, 'is': 11, 'best': 19}, {'Gfg': 6, 'is': 9, 'best': 10}, {'Gfg': 22, 'is': 6, 'best': 8}, {'Gfg': 12, 'is': 1, 'best': 8}]
The time complexity of this approach is O(nlogn) due to the use of sets and tuples.
Auxiliary space complexity is O(n) because we are creating a new list of dictionaries.
