Python Program to Remove Duplicate Elements From an Array

An array is a collection of elements of same data type, and each element in the array is identified by an index value. It is a simplest data structure where each data element can be accessed directly by only using its index number.

Arrays in Python

Python does not have a specific data structure to represent arrays. Here, we can use List an array.

 [6, 4, 1, 5, 9]
  0  1  2  3  4

The indexing in python starts from 0. In the above block the integers 6, 4, 1, 5, 9 are the array elements and 0, 1, 2, 3, 4 are the respective index values.

An array can have duplicate elements, in this article, we will discuss a few ways to remove duplicate elements from an array.

Input Output Scenarios

Assuming we have an input array with duplicate values. And the resultant array will have unique elements only.

Input array:
A = [1, 5, 3, 6, 3, 5, 6, 1]
Output array:
[1, 5, 3, 6]

The elements 1, 5, 3, 6 are the unique elements from the given array.

Using For Loop

We will use the for loop to iterate all array elements, in each iteration we will find the duplicates using the not in operator.

Example

In this example, firstly we initialize an empty list result to store all the unique values, which are found in the for loop.

lst = [1, 5, 3, 6, 3, 5, 6,  1] 
print ("The original array is: ",lst) 

# Remove repeated elements from array 
result = []

for i in lst: 
   if i not in result: 
      result.append(i) 

print ("The array after removing repeated elements: ", result)

Output

The original array is:  [1, 5, 3, 6, 3, 5, 6, 1]
The array after removing repeated elements:  [1, 5, 3, 6]

The "not in" operator is checking, if the current element exists in the empty list or not. If it does not exist, then that element is appended to the result list, else ignore the element.

Using Sets

Set is a data structure in python, which stores unique data. That means, it cannot allow storing of duplicate elements.

Example

In this example, we will simply do the type conversion of the array from list data type to set data type.

lst = [1, 5, 3, 6, 3, 5, 6,  1] 
print ("The original array is: ",lst) 

# Remove repeated elements from array 
result = list(set(lst)) 

print ("The array after removing repeated elements: ", result) 

Output

The original array is:  [1, 5, 3, 6, 3, 5, 6, 1]
The array after removing repeated elements:  [1, 3, 5, 6]

As we know that the set data structure cannot hold a duplicate item in it, so that we got output array with all the unique elements.

Using Enumerate() Function

Enumerate() is a python built-in function, it takes an iterable and returns a tuple containing a count and the values obtained from iterating over the iterable.

Syntax

enumerate(iterable, start=0)

Example

We will execute the enumerate() function in a list comprehension to keep a track on the index of each element of the array, then the index value i can be used to check if the element n is already exists in the array up to the index i. If it exist, we will ignore the element otherwise we will add it to the resultant array.

lst = [1, 5, 3, 6, 3, 5, 6,  1] 
print ("The original array is: ",lst) 

# Remove repeated elements from array 
result = [i for i, n in enumerate(lst) if n not in lst[:i]]

print ("The array after removing repeated elements: ", result) 

Output

The original array is:  [1, 5, 3, 6, 3, 5, 6, 1]
The array after removing repeated elements:  [1, 5, 3, 6]

Using Dict.fromkeys()

The python dict.fromkeys() method is used to create a dictionary from the given set of keys and values. A dictionary stores a unique set of keys.

Syntax

dict.fromkeys(keys, values)

Parameters

• Keys − it is a required parameter. It takes an iterable specifying the keys of the new dictionary.

• Values − it is an optional parameter, The value for all keys. The default value is None.

Example

In this example, we will create a dictionary with keys only, not with the key and value pairs.

lst = [1, 5, 3, 6, 3, 5, 6,  1] 
print ("The original array is: ",lst) 

# Remove repeated elements from array
 
result = list(dict.fromkeys(lst))

print ("The array after removing repeated elements: ", result) 

Output

The original array is:  [1, 5, 3, 6, 3, 5, 6, 1]
The array after removing repeated elements:  [1, 5, 3, 6]

As we know that the keys in a dictionary cannot be duplicated. Therefore, the fromkeys() method removes repeated values on its own. Then we converted it to a list to get the array with all unique elements.

These are a few of the aproches to which we can remove the repeated elements from an array.