How to Stack Multiple Pandas DataFrames?

Pandas provides several methods to stack multiple DataFrames vertically or horizontally. When working with multiple datasets that need to be combined for analysis, functions like concat(), append() (deprecated), and numpy.vstack() offer different approaches for DataFrame stacking.

This article explores the most effective methods for combining multiple DataFrames, focusing on practical examples and their use cases.

Syntax

pd.concat()

pd.concat(objs, axis=0, join='outer', keys=None, ignore_index=False)

Concatenates DataFrames along a specified axis with options for join types and index handling.

numpy.vstack()

numpy.vstack(tup)

Stacks arrays vertically (row-wise), provided they have the same number of columns.

Creating Sample DataFrames

First, let's create some sample DataFrames to work with ?

import pandas as pd
import numpy as np

# Create sample DataFrames
df1 = pd.DataFrame({
    'A': ['A0', 'A1', 'A2'],
    'B': ['B0', 'B1', 'B2']
})

df2 = pd.DataFrame({
    'A': ['A3', 'A4', 'A5'],
    'B': ['B3', 'B4', 'B5']
})

df3 = pd.DataFrame({
    'A': ['A6', 'A7'],
    'B': ['B6', 'B7']
})

print("DataFrame 1:")
print(df1)
print("\nDataFrame 2:")
print(df2)
print("\nDataFrame 3:")
print(df3)

DataFrame 1:
    A   B
0  A0  B0
1  A1  B1
2  A2  B2

DataFrame 2:
    A   B
0  A3  B3
1  A4  B4
2  A5  B5

DataFrame 3:
    A   B
0  A6  B6
1  A7  B7

Using pd.concat() with List of DataFrames

The most common and recommended method for stacking multiple DataFrames ?

import pandas as pd

# Create DataFrames
df1 = pd.DataFrame({'A': ['A0', 'A1', 'A2'], 'B': ['B0', 'B1', 'B2']})
df2 = pd.DataFrame({'A': ['A3', 'A4', 'A5'], 'B': ['B3', 'B4', 'B5']})
df3 = pd.DataFrame({'A': ['A6', 'A7'], 'B': ['B6', 'B7']})

# Stack multiple DataFrames
dataframes = [df1, df2, df3]
result = pd.concat(dataframes)

print(result)

    A   B
0  A0  B0
1  A1  B1
2  A2  B2
0  A3  B3
1  A4  B4
2  A5  B5
0  A6  B6
1  A7  B7

Using pd.concat() with Keys for Hierarchical Index

Add keys to create a hierarchical index that identifies the source DataFrame ?

import pandas as pd

# Create DataFrames
df1 = pd.DataFrame({'A': ['A0', 'A1', 'A2'], 'B': ['B0', 'B1', 'B2']})
df2 = pd.DataFrame({'A': ['A3', 'A4', 'A5'], 'B': ['B3', 'B4', 'B5']})

# Concatenate with keys
result = pd.concat([df1, df2], keys=['dataset1', 'dataset2'])

print(result)

           A   B
dataset1 0  A0  B0
         1  A1  B1
         2  A2  B2
dataset2 0  A3  B3
         1  A4  B4
         2  A5  B5

Using pd.concat() with ignore_index

Reset the index to create a continuous sequence ?

import pandas as pd

# Create DataFrames
df1 = pd.DataFrame({'A': ['A0', 'A1', 'A2'], 'B': ['B0', 'B1', 'B2']})
df2 = pd.DataFrame({'A': ['A3', 'A4', 'A5'], 'B': ['B3', 'B4', 'B5']})

# Concatenate with reset index
result = pd.concat([df1, df2], ignore_index=True)

print(result)

    A   B
0  A0  B0
1  A1  B1
2  A2  B2
3  A3  B3
4  A4  B4
5  A5  B5

Using numpy.vstack()

Convert DataFrames to numpy arrays and stack them vertically ?

import pandas as pd
import numpy as np

# Create DataFrames
df1 = pd.DataFrame({'A': ['A0', 'A1', 'A2'], 'B': ['B0', 'B1', 'B2']})
df2 = pd.DataFrame({'A': ['A3', 'A4', 'A5'], 'B': ['B3', 'B4', 'B5']})

# Stack using numpy.vstack
result_array = np.vstack([df1.values, df2.values])
result = pd.DataFrame(result_array, columns=df1.columns)

print(result)

    A   B
0  A0  B0
1  A1  B1
2  A2  B2
3  A3  B3
4  A4  B4
5  A5  B5

Comparison of Methods

Conclusion

Use pd.concat() for most DataFrame stacking operations as it's flexible and handles various data types. For purely numeric data requiring maximum performance, numpy.vstack() is the fastest option.