How to display notnull rows and columns in a Python dataframe?

In this tutorial, we will learn how to display notnull rows and columns in a Python dataframe with the help of some libraries. We will be using the Pandas library in this tutorial.

A dataframe is a data structure in pandas similar to an Excel sheet or a SQL table. It is a two-dimensional labeled data structure that can hold multiple columns of potentially different types of data such as integer, float, string, etc. Pandas provides a powerful data structure, "dataframe" and other useful methods to work on huge data.

Approach 1

One way to display the non-null rows and columns in a dataframe is to use the dropna() method. It returns a dataframe with all the columns and rows containing null values removed from the original dataframe.

Syntax

To display an image using the dropna() method, you need to follow the following syntax −

df = df.dropna()
print(df)

We use the ‘dropna()’ method on a dataframe 'df' for which we want to filter out all the null values. Then we will print it using the print() function.

Example

In this example, we are using the Pandas library. So first, we will import Pandas library and assign it an alias 'pd'. The dictionary named 'data' contains some sample data with its keys representing columns, and their values are lists representing the rows. In this, we intentionally made some values 'None' to demonstrate the use of the 'dropna()' method. Then we will pass this dictionary to the pandas DataFrame() function, which returns a dataframe object using 'pd.DataFrame(data)'.

Then we will use the dropna() method on this 'df' dataframe to drop null rows and columns. Then we will use the print() function to display the required dataframe.

import pandas as pd

# Create a sample dataframe with null values
data = {'Name': ['Alice', 'Bob', None, 'David', 'Eva'],
        'Age': [25, 30, None, 20, 28],
        'Gender': ['F', 'M', 'M', 'M', None],
        'City': [None, 'San Francisco', 'Boston', 'Los Angeles', None]}
df = pd.DataFrame(data)

# Drop rows and columns with null values
df = df.dropna()

# Display the resulting dataframe
print(df)

Output

    Name   Age Gender           City
1    Bob  30.0      M  San Francisco
3  David  20.0      M    Los Angeles

Example

In this example, we are creating a sample dataframe with some null values in it using the pandas's DataFrame() function. Then we use the 'dropna()' method on this dataframe to drop all the rows containing less than 2 non-null values by passing its 'thresh' parameter a value of 2.

We passed the 'inplace' parameter of the 'dropna()' function a value of 'True', which specifies to replace the original dataframe with the new dataframe in which non-null values are dropped. The 'inplace' parameter's default value is 'True'; in that case, it returns a dataframe and doesn't change the original one. Finally, we will display the resulting dataframe using the print() function.

import pandas as pd

# Create a sample dataframe with null values
data = {'Name': ['Alice', 'Bob', None, 'David', 'Eva'],
        'Age': [25, 30, None, 20, None],
        'Gender': ['F', 'M', 'M', 'M', None],
        'City': [None, 'San Francisco', 'Los Angeles', 'Boston', None]}
df = pd.DataFrame(data)

# Drop rows and columns with null values in-place
df.dropna(thresh=2, inplace=True)

# Display the resulting dataframe
print(df)

Output

    Name   Age Gender           City
0  Alice  25.0      F           None
1    Bob  30.0      M  San Francisco
2   None   NaN      M    Los Angeles
3  David  20.0      M         Boston

Approach 2

Another way to display the non-null values in the dataframe is to use the 'notnull()' method. It is a pandas function that returns a boolean dataframe of the same shape as of the original dataframe where 'True' represents a non-null value and 'False' represents a null value.

Syntax

To display the non-null values using the notnull() method, you need to follow the following syntax

df = df[df.notnull().all(axis=1)]
print(df)

We use the 'notnull()’ method on a dataframe 'df' in which we want to filter out the null values. The 'all()' method with its 'axis' parameter set to '1' only returns the rows which contain only non-null values. Then we will print the resulting dataframe using the print() function.

Example

In this example, we first create a custom dataframe with some null values in it. Then we use the 'notnull()' method on that dataframe to create a boolean dataframe where null values are represented as 'False' and the non-null values are represented as 'True'. Then we use the 'all()' method on this boolean dataframe with the parameter 'axis' set to '1', which returns only rows where all values are non-null.

Finally, we assign the resulting filtered dataframe back to the original dataframe variable 'df' and display the resulting dataframe using the print() function.

import pandas as pd

# Create a sample dataframe
data = {'Name': ['Alice', 'Bob', 'Charlie', 'David', 'Eva'],
        'Age': [25, 30, None, 20, 28],
        'Gender': ['F', 'M', 'M', 'M', None],
        'City': ['New York', 'San Francisco', 'Los Angeles', 'Boston', None]}
df = pd.DataFrame(data)

# Filter for rows and columns with non-null values
df = df[df.notnull().all(axis=1)]

# Display the resulting dataframe
print(df)

Output

    Name   Age Gender           City
0  Alice  25.0      F       New York
1    Bob  30.0      M  San Francisco
3  David  20.0      M         Boston

Conclusion

We learned how to use different methods to display the not null values in a Python dataframe. We learned how to create dataframes consisting of custom data by using the Pandas DataFrame() function. The different methods discussed are very useful for filtering rows and columns with non-null values, which can be useful when working with large datasets, as null values can often cause problems with data analysis and modeling. By removing these null values, we can ensure that our data is clean and accurate and that we are making the most of the available data.