Python – Remove Columns of Duplicate Elements

When it is required to remove columns of duplicate elements, a method is defined that creates an empty set. The list is iterated over, and if it is not found, it is added to the set.

Example

Below is a demonstration of the same

from itertools import chain

def remove_dupes(my_sub):

   my_string = set()
   for i, elem in enumerate(my_sub):

      if elem not in my_string:
         my_string.add(elem)
      else:
         yield i

my_list = [[5, 1, 6, 7, 9], [6, 3, 1, 9, 1], [4, 2, 9, 8, 9], [5, 1, 6, 7, 3]]

print("The list is : ")
print(my_list)

K = 3

temp_idxs = set(chain.from_iterable(remove_dupes(sub) for sub in my_list))

my_result = [[elem for i, elem in enumerate( sub) if i not in temp_idxs] for sub in my_list]

print("The result is : ")
print(my_result)

Output

The list is :
[[5, 1, 6, 7, 9], [6, 3, 1, 9, 1], [4, 2, 9, 8, 9], [5, 1, 6, 7, 3]]
The result is :
[[5, 1, 6, 7], [6, 3, 1, 9], [4, 2, 9, 8], [5, 1, 6, 7]]

Explanation

• The required packages are imported into the environment.

• A method named ‘remove_dupes’ is defined that takes a list as a parameter.

• An empty set is created.

• The elements in the list are enumerated over, and if an element is not found, it is added to the empty set.

• Otherwise, the iterator of the enumeration is yielded.

• Outside the function, a list of list is defined and is displayed on the console.

• A value for ‘K’ is defined.

• The method is called by iterating over it.

• It is converted to a set and is assigned to a variable.

• Again, it is enumerated, and iterated and stored in a result variable.

• This result variable is displayed as output on the console.