Duplicate substring removal from list in Python

Sometimes we may have a need to refine a given list by eliminating the duplicate elements in it. This can be achieved by using a combination of various methods available in python standard library.

with set and split

The split method can be used to segregate the elements for duplicate checking and the set method is used to store the unique elements from the segregated list elements.

Example

# initializing list
listA = [ 'xy-xy', 'pq-qr', 'xp-xp-xp', 'dd-ee']

print("Given list : ",listA)

# using set() and split()
res = [set(sub.split('-')) for sub in listA]

# Result
print("List after duplicate removal : " ,res)

Output

Running the above code gives us the following result −

Given list : ['xy-xy', 'pq-qr', 'xp-xp-xp', 'dd-ee']
List after duplicate removal : [{'xy'}, {'pq', 'qr'}, {'xp'}, {'ee', 'dd'}]

With list

We can also use the list method and use for loops along with it so that only unique elements from the list after segregations are captured.

Example

# initializing list
listA = [ 'xy-xy', 'pq-qr', 'xp-xp-xp', 'dd-ee']

print("Given list : ",listA)

# using list
res = list({i for sub in listA for i in sub.split('-')})

# Result
print("List after duplicate removal : " , res)

Output

Running the above code gives us the following result −

Given list : ['xy-xy', 'pq-qr', 'xp-xp-xp', 'dd-ee']
List after duplicate removal : ['dd', 'pq', 'ee', 'xp', 'xy', 'qr']

Pradeep Elance

Published on 26-Aug-2020 07:04:35

• Related Questions & Answers
• Remove duplicate tuples from list of tuples in Python
• Longest Duplicate Substring in C++
• Python prorgam to remove duplicate elements index from other list
• Python – All occurrences of Substring from the list of strings
• Python program to remove duplicate elements from a Circular Linked List
• Program to find duplicate item from a list of elements in Python
• Maximum removal from array when removal time >= waiting time in C++
• Python program – All occurrences of Substring from the list of strings
• Can Make Palindrome from Substring in Python
• Python program to remove duplicate elements from a Doubly Linked List\n
• Remove tuples having duplicate first value from given list of tuples in Python
• C# program to remove duplicate elements from a List
• Python – Merge Dictionaries List with duplicate Keys
• Program to remove duplicate entries in a list in Python
• C++ program to find array after removal of left occurrences of duplicate elements