How to display notnull rows and columns in a Python dataframe?

In this tutorial, we will learn how to display notnull rows and columns in a Python dataframe using the Pandas library. A dataframe is a two-dimensional labeled data structure that can hold multiple columns of potentially different data types such as integer, float, string, etc.

Using dropna() Method

The dropna() method returns a dataframe with all rows and columns containing null values removed from the original dataframe ?

Syntax

df.dropna()

Example

In this example, we create a sample dataframe with some null values and use dropna() to remove rows containing any null values ?

import pandas as pd

# Create a sample dataframe with null values
data = {'Name': ['Alice', 'Bob', None, 'David', 'Eva'],
        'Age': [25, 30, None, 20, 28],
        'Gender': ['F', 'M', 'M', 'M', None],
        'City': [None, 'San Francisco', 'Boston', 'Los Angeles', None]}
df = pd.DataFrame(data)

print("Original DataFrame:")
print(df)

# Drop rows with any null values
df_clean = df.dropna()

print("\nDataFrame after removing null rows:")
print(df_clean)

Original DataFrame:
    Name   Age Gender           City
0  Alice  25.0      F           None
1    Bob  30.0      M  San Francisco
2   None   NaN      M         Boston
3  David  20.0      M    Los Angeles
4    Eva  28.0   None           None

DataFrame after removing null rows:
    Name   Age Gender           City
1    Bob  30.0      M  San Francisco
3  David  20.0      M    Los Angeles

Using thresh Parameter

The thresh parameter allows you to keep rows that have at least a specified number of non-null values ?

import pandas as pd

# Create a sample dataframe with null values
data = {'Name': ['Alice', 'Bob', None, 'David', 'Eva'],
        'Age': [25, 30, None, 20, None],
        'Gender': ['F', 'M', 'M', 'M', None],
        'City': [None, 'San Francisco', 'Los Angeles', 'Boston', None]}
df = pd.DataFrame(data)

print("Original DataFrame:")
print(df)

# Keep rows with at least 3 non-null values
df_thresh = df.dropna(thresh=3)

print("\nDataFrame with at least 3 non-null values per row:")
print(df_thresh)

Original DataFrame:
    Name   Age Gender           City
0  Alice  25.0      F           None
1    Bob  30.0      M  San Francisco
2   None   NaN      M    Los Angeles
3  David  20.0      M         Boston
4    Eva   NaN   None           None

DataFrame with at least 3 non-null values per row:
    Name   Age Gender           City
0  Alice  25.0      F           None
1    Bob  30.0      M  San Francisco
2   None   NaN      M    Los Angeles
3  David  20.0      M         Boston

Using notnull() Method

The notnull() method returns a boolean dataframe where True represents non-null values and False represents null values ?

Syntax

df[df.notnull().all(axis=1)]

Example

We use notnull().all(axis=1) to filter rows where all columns contain non-null values ?

import pandas as pd

# Create a sample dataframe
data = {'Name': ['Alice', 'Bob', 'Charlie', 'David', 'Eva'],
        'Age': [25, 30, None, 20, 28],
        'Gender': ['F', 'M', 'M', 'M', None],
        'City': ['New York', 'San Francisco', 'Los Angeles', 'Boston', None]}
df = pd.DataFrame(data)

print("Original DataFrame:")
print(df)

# Filter for rows with all non-null values
df_filtered = df[df.notnull().all(axis=1)]

print("\nRows with all non-null values:")
print(df_filtered)

Original DataFrame:
      Name   Age Gender           City
0    Alice  25.0      F       New York
1      Bob  30.0      M  San Francisco
2  Charlie   NaN      M    Los Angeles
3    David  20.0      M         Boston
4      Eva  28.0   None           None

Rows with all non-null values:
    Name   Age Gender           City
0  Alice  25.0      F       New York
1    Bob  30.0      M  San Francisco
3  David  20.0      M         Boston

Comparison

Conclusion

Use dropna() for straightforward removal of null values. For more control, use thresh parameter or notnull() with boolean indexing. These methods ensure clean data for analysis and modeling.