Python Pandas - Read CSV

---

---

Reading data from a CSV (Comma-Separated Values) file is one of the most common tasks in data analysis and data science. Python's Pandas library provides a flexible read_csv() method for reading data from CSV files into a DataFrame.

This method simplifies the process of loading the data from a CSV file into a DataFrame (a 2D table-like structure with labeled rows and columns).

In this tutorial, we will learn various aspects of reading CSV files with Pandas, including advanced features like controlling the number of rows to read, parsing dates, handling missing data, and more.

Introduction to read_csv() Method

The pandas.read_csv() method in the Pandas library used to read a CSV file and converts the data into a Pandas DataFrame object. It can read files stored locally or remotely using supported URL schemes like HTTP, FTP, or cloud storage paths (e.g., S3).

This method supports a wide range of parameters for customizing headers, delimiters, and data types.

Key Features of read_csv() Method −

• This method can handle large datasets efficiently.

• It supports different delimiters and is not limited to reading comma-separated values.

• It supports reading data from different sources like, URLs, local files, and file-like objects.

• This method handles missing values effectively.

Syntax

Below is the syntax of the read_csv() method −

pandas.read_csv(filepath_or_buffer, sep=',', header='infer', names=None, index_col=None, usecols=None, dtype=None, ...)

Here are some commonly used parameters of this method −

• filepath_or_buffer: The file path, URL, or file-like object to read CSV file from.

• sep or delimiter: Specifies the character or regex pattern used to separate values in the file. Default is , (comma).

• header: Specifies the row number to use as column names.

• names: A list of column names to use when there is no header row.

Example

Here is a basic example of using the pandas.read_csv() method for reading a CSV data from local file.

data.csv

Name,Salary
Ravi,50000
Priya,45000
Kiran,65000
,55000

main.py

import pandas as pd

# Read a CSV file into a DataFrame
data = pd.read_csv('data.csv')

# Display the first few rows
print(data.head())

Output

Following is the output of the above code −

    Name  Salary
0   Ravi   50000
1  Priya   45000
2  Kiran   65000
3    NaN   55000

Reading Large CSV Files in Chunks

For reading large CSV files in small blocks, you can use the chunksize parameter of the read_csv() method. This parameter reads data into smaller, more manageable blocks of given size.

Example

This example shows loading the CSV data into the Pandas DataFrame of smaller chunks using the chunksize parameter.

import pandas as pd

url ="https://raw.githubusercontent.com/Opensourcefordatascience/Data-sets/master/blood_pressure.csv"

# Read the CSV data in chunks
chunks = pd.read_csv(url, chunksize=50)

for chunk in chunks:
   print("Output DataFrame:")
   print(chunk.head())

Output

When we run above program, it will produces following result −

Output DataFrame:
   patient   sex agegrp  bp_before  bp_after
0        1  Male  30-45        143       153
1        2  Male  30-45        163       170
2        3  Male  30-45        153       168
3        4  Male  30-45        153       142
4        5  Male  30-45        146       141
Output DataFrame:
    patient   sex agegrp  bp_before  bp_after
50       51  Male    60+        175       146
51       52  Male    60+        175       160
52       53  Male    60+        172       175
53       54  Male    60+        173       163
54       55  Male    60+        170       185
Output DataFrame:
     patient     sex agegrp  bp_before  bp_after
100      101  Female    60+        168       178
101      102  Female    60+        142       141
102      103  Female    60+        147       149
103      104  Female    60+        148       148
104      105  Female    60+        162       138

Reading CSV Files with Custom Delimiters

Sometimes, data in a CSV file may use a delimiter other than a comma. In that case you can use the sep or delimiter parameter to specify the custom character used as a separator.

Example

Here is an example that demonstrates specifying the custom delimiter to the sep parameter of the pandas.read_csv() method.

import pandas as pd

# Import StringIO to load a file-like object for reading CSV data
from io import StringIO

# Create a tab-delimited data
data = """
Sr.no\tName\tGender\tAge
1\tChinmayi\tfemale\t22
2\tMadhuri\tfemale\t38
3\tKarthik\tmale\t26
4\tGeetha\tfemale\t35
"""
# Use StringIO to convert the string data into a file-like object
obj = StringIO(data)

# Read a tab-separated data using read_csv()
data = pd.read_csv(obj, sep='\t')
print('Output DataFrame:')
print(data.head())

Output

The output of the above code as follows −

Output DataFrame:
   Sr.no      Name  Gender  Age
0      1  Chinmayi  female   22
1      2   Madhuri  female   38
2      3   Karthik    male   26
3      4    Geetha  female   35

Control Reading Number of Rows from CSV

You can limit the number of rows read from a CSV file using the nrows parameter of the pandas.read_csv() method.

Example

This example demonstrates how to limit the number of rows while reading the CSV data using the pandas.read_csv() method with the nrows parameter.

import pandas as pd
from io import StringIO

# Create a CSV data
data = """
Sr.no,Name,Gender,Age
1,Chinmayi,female,22
2,Madhuri,female,38
3,Karthik,male,26
4,Geetha,female,35
"""

# Use StringIO to convert the string data into a file-like object
obj = StringIO(data)

# Read a tab-separated data using read_csv()
data = pd.read_csv(obj, nrows=2)

print('Output DataFrame:')
print(data)

Output

Following is the output of the above code −

Output DataFrame:
   Sr.no      Name  Gender  Age
0      1  Chinmayi  female   22
1      2   Madhuri  female   38

Parsing Dates while Reading CSV Data

The parse_dates parameter handles datetime object while reading the CSV data containing the date columns.

Example

Here is an example that shows how pandas automatically parse date columns into datetime64 objects.

import pandas as pd

# Creating DataFrame 
dataFrame = pd.DataFrame({'Car': ['BMW', 'Lexus', 'Audi', 'Mercedes', 'Jaguar', 'Bentley'],
'Date_of_purchase': ['2024-10-10', '2024-10-12', '2024-10-17', '2024-10-16', '2024-10-19', '2024-10-22']})

# write dataFrame to CSV file
dataFrame.to_csv("Sample_CSV_File.csv")

# Parsing date columns
data = pd.read_csv("Sample_CSV_File.csv", parse_dates=['Date_of_purchase'])
print(data.info())

Output

The output of the above code as follows −

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 6 entries, 0 to 5
Data columns (total 3 columns):
 #   Column            Non-Null Count  Dtype         
---  ------            --------------  -----         
 0   Unnamed: 0        6 non-null      int64         
 1   Car               6 non-null      object        
 2   Date_of_purchase  6 non-null      datetime64[ns]
dtypes: datetime64[ns](1), int64(1), object(1)
memory usage: 276.0+ bytes