Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Physics
Chemistry
Biology
Mathematics
English
Economics
Psychology
Social Studies
Fashion Studies
Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
Merging duplicates to list of lists
At hand is the challenge of Python lists - an incredibly flexible data structure capable of holding various kinds of information when dealing with lists within lists (where multiple duplicate entries could exist) duplication is likely. Therefore a Pythonic solution must exist that removes duplicated sublists to ensure each sublist in a main list is unique. In this article, we illustrate how to merge duplicates to a list of lists with detailed examples using different approaches.
Merging Duplicates to a List of Lists by Using for Loop
Code Explanation and Design Steps −
Step 1 − Open Jupyter Notebook in Anaconda prompt and start writing the code in its cell.
Step 2 − In Python, we have an unordered list with sublists that contain duplicate items; our aim is to delete these duplicate items so each sublist in our main list will have only unique sublists.
Step 3 − Create a function called ‘merge_dups()’, takes an input list, processes it to remove duplicate sublists, and returns an output list without duplicate entries.
Step 4 − Within this function, we create an empty list that I call ‘output_list’; we use this space for sublists from our input list after checking whether there are duplicates.
Step 5 − Next, we create a ‘for’ loop to traverse each sublist within our input list, making this step essential to detecting potential duplicates individually in each sublist.
Step 6 − Within this loop, an if statement checks to see whether a sublist that we currently are looking at already appears within ‘output_list’ or needs to be added using ‘append()’ method; otherwise we simply advance forward until another one comes along.
Step 7 − Within this loop, an ‘if’ statement checks to see whether a sublist that we currently are looking at already appears within ‘output_list’ or needs to be added using ‘append()’ method; otherwise we simply advance forward until another one comes along.
Step 8 − Once this function has processed each sublist within its ‘input list’, ‘output_list’ should contain only unique sublists from that input list and any duplicative entries should have been eliminated from it.
Step 9 − At the core of our Python code is ‘my_list’ - an empty ‘test list’ which contains duplicate sublists which we hope to clear away by cleaning. This will serve as our target list to be decluttered.
Step 10 − Next, ‘merge_dups()’ is called on ‘my_list’ before assigning back its result variable for easy reuse in future code refactors. Since our function does not actually change our original list but rather generates one from scratch, to keep our code uncluttered we reuse its initial variable rather than change its contents further.
Step 11 − Finally, we print ‘my_list’ as the output from our function to verify its results. All sublists should be unique without duplicates appearing anywhere within them.
Example 1
Code for merging duplicates to list of lists −
def merge_dups(input_list):
output_list = []
for each_sublist in input_list:
if each_sublist not in output_list:
output_list.append(each_sublist)
return output_list
my_list = [[3, 10], [8, 2], [3, 10], [15, 6], [3, 17], [7, 8], [3,10], [15,6]]
my_list = merge_dups(my_list)
print(my_list)
Output
[[3, 10], [8, 2], [15, 6], [3, 17], [7, 8]]
Merging Duplicates to List of Lists Using Built-in Python Function and Data Type
In this article, we will tackle this problem using an alternative approach to what has been provided. The goal will remain unchanged, that of eliminating duplicate sublists from our main list by making use of Python's built-in data types and capabilities more efficiently than before.
Code Explanation and Design Steps −
Step 1 − Open Jupyter Notebook in Anaconda prompt and start writing the code in its cell.
Step 2 − Establish the Function − Create our function ‘remove_duplicates’ which accepts as it is input data source a list with potential duplicate sublists (input_list).
Step 3 − Convert Sublists to Tuples − This function begins by applying ‘map()'s Tuple function against each element within ‘input_list’; effectively turning every sublist into its respective Tuple.
Step 4 − Utilizing Sets to Eliminate Duplicates − Next we create a Python set from our list of tuples; as unique elements can only ever fit within it, duplicate tuples will automatically be eliminated when this ‘unique_tuple_set’ is created resulting in only distinct elements being contained by its members (unique_tuple_set).
Step 5 − Return Tuples Back Into Lists − Once we have created our set of unique tuples, the next step should be converting each individual one back into lists using list comprehension, an iterator that applies its list function back onto every unique tuple in our set so as to convert back into lists.
Step 6 − Return the Final List − Once all duplicates have been deleted and converted back into their appropriate formats (list of lists), this function returns the unique list as the result.
Step 7 − Utilization − With this function in our arsenal, we can now employ it on any list to eradicate duplicate sublists as shown here by creating ‘my_list’ with duplicate items, calling our function ‘remove_duplicates()’ with it as its argument and overwriting ‘my_list’ with its output list after calling ‘delete_all()’ with it as it’s first argument eventually printing ‘my_list’ to verify all duplicates have been eliminated successfully.
Example 2
Code for merging duplicates to a list of lists using built-in function and data type −
def remove_duplicates(input_list): tuple_list = map(tuple, input_list) unique_tuple_set = set(tuple_list) unique_list = [list(t) for t in unique_tuple_set] return unique_list my_list = [[3, 10], [8, 2], [3, 10], [15, 6], [3, 17], [7, 8], [3,10], [15,6]] my_list = remove_duplicates(my_list) print(my_list)
Output
[[3, 17], [3, 10], [15, 6], [8, 2], [7, 8]]
Conclusion
This article, uses two examples with different methods, the ways to show how to merge duplicates to a list of lists using Python language. By understanding and taking advantage of both sets, we are able to swiftly come up with effective answers.
56 Views
