Data Structure
Networking
RDBMS
Operating System
Java
MS Excel
iOS
HTML
CSS
Android
Python
C Programming
C++
C#
MongoDB
MySQL
Javascript
PHP
Physics
Chemistry
Biology
Mathematics
English
Economics
Psychology
Social Studies
Fashion Studies
Legal Studies
- Selected Reading
- UPSC IAS Exams Notes
- Developer's Best Practices
- Questions and Answers
- Effective Resume Writing
- HR Interview Questions
- Computer Glossary
- Who is Who
How to display all rows from dataframe using Pandas?
Pandas is a powerful and popular data manipulation library in Python that provides a flexible and efficient way to handle and analyze data. One of the key features of Pandas is its DataFrame object, which is a two-dimensional tabular data structure similar to a spreadsheet or a SQL table.
When printing a Pandas DataFrame directly in a Jupyter notebook or a Python console, it automatically truncates the display output when the DataFrame has many rows. By default, only a limited number of rows and columns are displayed to ensure that the output is concise and easier to read. This default behavior can be changed by modifying the max_rows and max_columns options of the Pandas module.
Displaying all Rows Using to_string() or to_markdown Function
In pandas, the to_string() function is used to display the complete dataframe, even if it contains a large number of rows or columns that exceed the maximum display limit. By default, pandas will truncate the output of a dataframe when it contains more rows or columns than the specified maximum display limit. However, the to_string() function allows displaying all the rows of a dataframe, regardless of their number.
The to_markdown() function in Pandas can be used to convert a dataframe to a Markdown-formatted table that can be easily displayed in a Jupyter notebook, a website, or a README file. The resulting table is easy to read and understand, as the headers and values are separated by pipes and aligned for better readability. Additionally, the to_markdown() function supports several options that allow customization of the output, such as hiding the index, specifying the alignment, and customizing the table headers.
Syntax
To display all rows from a dataframe using Pandas, you need to follow the following syntax −
print(pandas.dataFrameName.to_string()) OR print(pandas.dataFrameName.to_markdown())
To use the to_string() function, simply call it on the dataframe and use the print() function to display the resulting text. This method allows for customization of the output, including setting the maximum number of rows to display, specifying the column widths, and choosing how missing values are displayed.
To use the to_markdown() function, simply call it on the dataframe and use the print() function to display the resulting text. This method also displays all the rows in a better-formatted way.
Example
This code imports CSV data to a Pandas DataFrame called cricketers_df, which contains the details of 20 retired cricketers. Each row of the DataFrame represents a cricketer and contains their name, country, debut year, and retirement year.
Finally, the entire DataFrame is printed to the console using the to_string() method. This method converts the DataFrame to a string representation that can be printed to the console. Here we can see all the rows of the dataframe because of the to_string() method.
import pandas as pd
cricketers_df = pd.read_csv('cricketers.csv')
print(cricketers_df.to_string())
Output
Name Country Debut Year Retirement Year 0 Sachin Tendulkar India 1989 2013 1 Brian Lara West Indies 1990 2007 2 Ricky Ponting Australia 1995 2012 3 Jacques Kallis South Africa 1995 2014 4 Inzamam-ul-Haq Pakistan 1991 2007 5 Sanath Jayasuriya Sri Lanka 1989 2011 6 Muttiah Muralitharan Sri Lanka 1992 2010 7 Shane Warne Australia 1992 2007 8 Glenn McGrath Australia 1993 2007 9 Anil Kumble India 1990 2008 10 Sourav Ganguly India 1992 2008 11 Rahul Dravid India 1996 2012 12 VVS Laxman India 1996 2012 13 Adam Gilchrist Australia 1996 2008 14 Matthew Hayden Australia 1994 2009 15 Mark Waugh Australia 1991 2002 16 Steve Waugh Australia 1985 2004 17 Michael Vaughan England 1999 2009 18 Andrew Flintoff England 1998 2009 19 Marcus Trescothick England 2000 2008 20 Kevin Pietersen England 2004 2018
Example
In this example, we have used the same data and read the data from a pre-existing CSV file. This time, we print the whole data using the to_markdown() method on the dataframe.
import pandas as pd
cricketers_df = pd.read_csv('cricketers.csv')
print(cricketers_df.to_markdown())
Output
| | Name | Country | Debut Year | Retirement Year | |---:|:---------------------|:-------------|-------------:|------------------:| | 0 | Sachin Tendulkar | India | 1989 | 2013 | | 1 | Brian Lara | West Indies | 1990 | 2007 | | 2 | Ricky Ponting | Australia | 1995 | 2012 | | 3 | Jacques Kallis | South Africa | 1995 | 2014 | | 4 | Inzamam-ul-Haq | Pakistan | 1991 | 2007 | | 5 | Sanath Jayasuriya | Sri Lanka | 1989 | 2011 | | 6 | Muttiah Muralitharan | Sri Lanka | 1992 | 2010 | | 7 | Shane Warne | Australia | 1992 | 2007 | | 8 | Glenn McGrath | Australia | 1993 | 2007 | | 9 | Anil Kumble | India | 1990 | 2008 | | 10 | Sourav Ganguly | India | 1992 | 2008 | | 11 | Rahul Dravid | India | 1996 | 2012 | | 12 | VVS Laxman | India | 1996 | 2012 | | 13 | Adam Gilchrist | Australia | 1996 | 2008 | | 14 | Matthew Hayden | Australia | 1994 | 2009 | | 15 | Mark Waugh | Australia | 1991 | 2002 | | 16 | Steve Waugh | Australia | 1985 | 2004 | | 17 | Michael Vaughan | England | 1999 | 2009 | | 18 | Andrew Flintoff | England | 1998 | 2009 | | 19 | Marcus Trescothick | England | 2000 | 2008 | | 20 | Kevin Pietersen | England | 2004 | 2018 |
We learned how to display all rows from a dataframe using to_string() and to_markdown() functions in Pandas. The to_string() function is a useful tool for working with large dataframes, as it allows for easy visualization and analysis of all the data in the dataframe.
Unlike to_string(), which simply outputs the DataFrame as a string, to_markdown() outputs a markdown table that can be used in a document. The resulting table is easy to read and looks professional.
Displaying all Rows by Changing default Print Settings
Pandas's option_context() function allows temporarily modifying the display options for a dataframe within a specified context. By using this function, we can change various display options, such as the maximum number of rows and columns to display, the display width of each column, and the precision of float values. Using option_context(), we can temporarily modify the display options to view all rows or columns without having to permanently change the options.
In Pandas, set_option() is a method that allows you to set different options for the way data is displayed. One useful option is the display.max_rows option controls the maximum number of rows displayed in a DataFrame. By setting this value to None, you can display all the rows in the DataFrame, making it easier to view and analyze the data.
Syntax
To display all rows from a dataframe using Pandas by changing default settings, you need to follow the following syntax:
with pandas.option_context('display.max_rows', None,):
print(dataframeName)
The syntax provided sets the display option to show all the rows in a Pandas dataframe named 'dataframeName'. The option_context() function temporarily sets the maximum number of rows that can be displayed to None, which means that all the rows in the dataframe will be displayed when the dataframe is printed.
pandas.set_option('display.max_rows', None)
print(dataframeName)
To use set_option(), simply pass in the option you want to set, along with its value, like this: pd.set_option('display.max_rows', None). Once the option is set, any subsequent DataFrame outputs will reflect the changes you made. Changes made by this function are permanent until changed again.
Example
This code reads a CSV file fruits.csv, and stores its data in fruits_df, which contains data about various fruits, including their names, colors, weights, and prices. The DataFrame has 15 entries, with each entry representing a different fruit.
The code then uses the option_context() function to temporarily set the maximum number of rows to display in the DataFrame to None. This means that all rows of the DataFrame will be displayed when it is printed, rather than just a summary.
Finally, the code prints the fruits_df DataFrame using print(fruits_df), which will display all 15 rows due to the option_context() setting.
import pandas as pd
fruits_df = pd.read_csv('fruits.csv')
with pd.option_context('display.max_rows', None,):
print(fruits_df)
Output
Name Color Weight (oz) Price ($) 0 Apple Red 4.0 0.50 1 Orange Orange 6.0 0.40 2 Banana Yellow 5.0 0.20 3 Grapes Green 3.0 0.30 4 Pineapple Brown 16.0 1.50 5 Strawberry Red 1.0 0.10 6 Watermelon Green 128.0 2.00 7 Kiwi Brown 3.0 0.30 8 Mango Orange 8.0 1.00 9 Pear Green 5.0 0.40 10 Peach Orange 6.0 0.50 11 Plum Purple 4.0 0.40 12 Cherry Red 0.5 0.10 13 Blueberry Blue 0.3 0.05 14 Raspberry Red 0.2 0.05
Example
In this example, we have used the same fruit data by reading it from a CSV file. Then, we used the set_option() to alter the default settings and print the whole data.
import pandas as pd
fruits_df = pd.read_csv('fruits.csv')
pd.set_option('display.max_rows', None)
print(fruits_df)
Output
Name Color Weight (oz) Price ($) 0 Apple Red 4.0 0.50 1 Orange Orange 6.0 0.40 2 Banana Yellow 5.0 0.20 3 Grapes Green 3.0 0.30 4 Pineapple Brown 16.0 1.50 5 Strawberry Red 1.0 0.10 6 Watermelon Green 128.0 2.00 7 Kiwi Brown 3.0 0.30 8 Mango Orange 8.0 1.00 9 Pear Green 5.0 0.40 10 Peach Orange 6.0 0.50 11 Plum Purple 4.0 0.40 12 Cherry Red 0.5 0.10 13 Blueberry Blue 0.3 0.05 14 Raspberry Red 0.2 0.05
Conclusion
We learned how to display all rows from a dataframe by changing the default print settings in Pandas. The option_context() function can help to improve the readability and usability of dataframes and make it easier to analyze large datasets. The modified display options are only active within the specified context and return to the original settings once the context is exited.
The set_option() method is as useful as option_context(), but the only drawback here is that it makes permanent changes to the print settings until changed again.
9K+ Views
