Merge, Join and Concatenate DataFrames using Pandas

In this tutorial, we are going to learn to merge, join, and concat the DataFrames using pandas library. I think you are already familiar with dataframes and pandas library. Let's see the three operations one by one.

Merge

We have a method called pandas.merge() that merges dataframes similar to the database join operations. Follow the below steps to achieve the desired output. Merge method uses the common column for the merge operation.

• Initialize the Dataframes.

• Call the method pandas.merge() with three arguments dataframes, how (defines the database join operation), on (common field of the dataframes).

Example

Let's see an example.

# importing the pandas library
import pandas
# creating dataframes
    dataframe_1 = pandas.DataFrame({"Common": ["A", "B", "C", "D", "E"],
   "Name": ["John", "Alice", "Emma", "Watson", "Harry"], "Age": [18, 19, 20, 21, 15]}) dataframe_2 =     pandas.DataFrame({"Common": ["A", "B", "C", "D", "E"], "Sport": ["Cricket", "Football", "Table     Tennis", "Badminton", "Chess"], "Movie": ["Jumanji", "Black Widow", "End Game", "Mr. Robot",          "Matrix"]})
# merging using merge method
# how = left or right or inner
new_df = pandas.merge(dataframe_1, dataframe_2, how="left", on="Common") # printing the resultant dataframe print(new_df)

Output

If you run the above code, you will get the following results.

Common Name   Age  Sport            Movie
0   A  John   18  Cricket         Jumanji
1   B  Alice  19  Football    Black Widow
2   C  Emma   20  Table Tennis   End Game
3   D  Watson 21  Badminton     Mr. Robot
4   E  Harry  15  Chess            Matrix

Join

Similar to the merge method, we have a method called dataframe.join(dataframe) for joining the dataframes. Let's see steps to join two dataframes into one. The join method uses the index of the dataframe.

• Initialize the dataframes.

• Write a statment dataframe_1.join(dataframe_2) to join.

Example

Let's try it with the coding example.

# importing the pandas library
import pandas
# creating dataframes
   dataframe_1 = pandas.DataFrame({"Name": ["John", "Alice", "Emma", "Watson", "Harry"], "Age": [18,    19, 20, 21, 15]}, index = ["A", "B", "C", "D", "E"])dataframe_2 = pandas.DataFrame({"Sport":          ["Cricket", "Football", "Table Tennis", "Badminton", "Chess"], "Movie": ["Jumanji", "Black Widow",    "End Game", "Mr. Robot", "Matrix"]}, index = ["A", "B", "C", "D", "E"])
   # joining
   new_df = dataframe_1.join(dataframe_2)
# printing the new dataframe
print(new_df)

If you run the above program, you will get the following output

Output

     Name   Age   Sport           Movie
A    John   18   Cricket        Jumanji
B    Alice  19   Football   Black Widow
C    Emma   20   Table Tennis  End Game
D    Watson  21  Badminton    Mr. Robot
E    Harry   15   Chess          Matrix

Concatenation

Similar to the merge and join methods, we have a method called pandas.concat(list->dataframes) for concatenation of dataframes. Let's see steps to concatenate dataframes. Concatenation combines dataframes into one.

• Initialize the dataframes.

• Concatenate dataframes using pandas.concat([df_1, df_2, ..]). Print the result.

Example

Let's try it with the coding example.

# importing the pandas library
import pandas
   # creating dataframes dataframe_1 = pandas.DataFrame({"Name":                                        ["John","Alice","Emma","Watson","Harry"], "Age": [18, 19, 20, 21, 15]}, index = ["A", "B", "C",       "D", "E"]) dataframe_2 = pandas.DataFrame({"Name": ["Wick", "Robert", "Elliot", "Baby",                "Cruise"], "Age": [22, 20, 45, 15, 42]}, index = ["F", "G", "H", "I", "J"])
   # concatenating -> you can pass any number of new_df = pandas.concat([dataframe_1, dataframe_2])
# printing the new dataframe
print(new_df)

Output

If you run the above program, you will get the following output.

   Name    Age
A  John     18
B Alice     19
C Emma      20
D Watson    21
E Harry     15
F Wick      22
G Robert    20
H Elliot    45
I Baby      15
J Cruise    42

Conclusion

If you have any doubts in the tutorial, mention them in the comment section.