Python Pandas – Fetch the Common rows between two DataFrames with concat()

To fetch the common rows between two DataFrames, use the concat() function. Let us create DataFrame1 with two columns −

dataFrame1 = pd.DataFrame(
   {
      "Car": ['BMW', 'Lexus', 'Audi', 'Tesla', 'Bentley', 'Jaguar'],
      "Reg_Price": [1000, 1500, 1100, 800, 1100, 900] }
)

Create DataFrame2 with two columns −

dataFrame2 = pd.DataFrame(
   {
"Car": ['BMW', 'Lexus', 'Audi', 'Tesla', 'Bentley', 'Jaguar'],
"Reg_Price": [1200, 1500, 1000, 800, 1100, 1000]
}
)

Finding common rows between two DataFrames with concat() −

dfRes = pd.concat([dataFrame1, dataFrame2])

Reset index −

dfRes = dfRes.reset_index(drop=True)

Groupby columns −

dfGroup = dfRes.groupby(list(dfRes.columns))

Getting the length of each row to calculate the count. If count is greater than 1, that would mean common rows −

res = [k[0] for k in dfGroup.groups.values() if len(k) > 1]

Example

Following is the code −

import pandas as pd

# Create DataFrame1
dataFrame1 = pd.DataFrame(
{
"Car": ['BMW', 'Lexus', 'Audi', 'Tesla', 'Bentley', 'Jaguar'],
"Reg_Price": [1000, 1500, 1100, 800, 1100, 900] }
)

print"DataFrame1 ...\n",dataFrame1

# Create DataFrame2
dataFrame2 = pd.DataFrame(
{
"Car": ['BMW', 'Lexus', 'Audi', 'Tesla', 'Bentley', 'Jaguar'],
"Reg_Price": [1200, 1500, 1000, 800, 1100, 1000]
}
)

print"\nDataFrame2 ...\n",dataFrame2

# finding common rows between two DataFrames
dfRes = pd.concat([dataFrame1, dataFrame2])

# reset index
dfRes = dfRes.reset_index(drop=True)

# groupby columns
dfGroup = dfRes.groupby(list(dfRes.columns))

# length of each row to calculate the count
# if count is greater than 1, that would mean common rows
res = [k[0] for k in dfGroup.groups.values() if len(k) > 1]

print"\nCommon rows...\n",dfRes.reindex(res)

Output

This will produce the following output −

DataFrame1 ...
       Car   Reg_Price
0      BMW        1000
1    Lexus        1500
2     Audi        1100
3    Tesla         800
4  Bentley        1100
5   Jaguar         900

DataFrame2 ...
       Car   Reg_Price
0      BMW        1200
1    Lexus        1500
2     Audi        1000
3    Tesla         800
4  Bentley        1100
5   Jaguar        1000

Common rows...
       Car   Reg_Price
3    Tesla         800
1    Lexus        1500
4  Bentley        1100

AmitDiwan

Updated on 14-Sep-2021 14:24:38

• Related Questions & Answers
• Python Pandas – Find the common rows between two DataFrames with merge()
• Python Pandas - Finding the uncommon rows between two DataFrames
• Python - Fetch columns between two Pandas DataFrames by Intersection
• Python Pandas – Find the common rows between two Data Frames
• Python Pandas – Find the Difference between two Dataframes
• Python Pandas – Can we use & Operator to find common columns between two DataFrames?
• Python - How to Concatenate Two or More Pandas DataFrames along rows?
• Fetch data between two rows in MySQL?
• How to compare two DataFrames in Python Pandas with missing values
• Python - How to Concatenate more than two Pandas DataFrames?
• How to append two DataFrames in Pandas?
• Python Pandas – Check if two Dataframes are exactly same
• Select DataFrame rows between two index values in Python Pandas
• Python – Get the Columns Shared by Two Pandas DataFrames using Numpy
• Python Pandas - Iterate and fetch the rows that contain the desired text