Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Physics
Chemistry
Biology
Mathematics
English
Economics
Psychology
Social Studies
Fashion Studies
Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
How to Merge all CSV Files into a single dataframe – Python Pandas?
To merge all CSV files, use the GLOB module. The os.path.join() method is used inside the concat() to merge the CSV files together.
At first, import the required libraries. We have set pd as an alias for the pandas library −
import pandas as pd import glob import os
Now, let’s say we have the following 3 CSV Files −
Sales1.csv
Sales2.csv
Sales3.csv
At first, set the path for joining multiple files. We have all the CSV files to be merged on the Desktop −
files = os.path.join("C:\Users\amit_\Desktop\", "sales*.csv")
Next, use glob to return the list of merged files −
files = glob.glob(files)
Example
Following is the code −
import pandas as pd
import glob
import os
# setting the path for joining multiple files
files = os.path.join("C:\Users\amit_\Desktop\", "sales*.csv")
# list of merged files returned
files = glob.glob(files)
print("Resultant CSV after joining all CSV files at a particular location...");
# joining files with concat and read_csv
df = pd.concat(map(pd.read_csv, files), ignore_index=True)
print(df)Output
This will produce the following −
Resultant CSV after joining all CSV files at a particular location... Car Place UnitsSold 0 Audi Bangalore 80 1 Porsche Mumbai 110 2 RollsRoyce Pune 100 3 BMW Delhi 95 4 Mercedes Hyderabad 80 5 Lamborghini Chandigarh 80 6 Volvo Rajasthan 150 7 Hyundai Manipur 120 8 Toyota HP 70
Updated on: 04-Oct-2021
16K+ Views
Advertisements