Selecting with complex criteria from a Pandas DataFrame

We can use different criteria to compare all the column values of a Pandas DataFrame. We can perform comparison operations like df[col] < 5, df[col] == 10, etc. For example, if we use the criteria df[col] > 2, then it will check all the values from col and compare whether they are greater than 2. For all the column values, it will return True if the condition holds, else False.

Basic Comparison Operations

Example

Let's create a DataFrame and apply various comparison criteria ?

import pandas as pd

df = pd.DataFrame(
    {
        "x": [5, 2, 7, 0],
        "y": [4, 7, 5, 1],
        "z": [9, 3, 5, 1]
    }
)
print("Input DataFrame is:")
print(df)

col = "x"
print(f"\nElements > 5 in column {col}:")
print(df[col] > 5)
print(f"\nElements == 5 in column {col}:")
print(df[col] == 5)

col = "y"
print(f"\nElements < 5 in column {col}:")
print(df[col] < 5)
print(f"\nElements != 5 in column {col}:")
print(df[col] != 5)

Input DataFrame is:
   x  y  z
0  5  4  9
1  2  7  3
2  7  5  5
3  0  1  1

Elements > 5 in column x:
0    False
1    False
2     True
3    False
Name: x, dtype: bool

Elements == 5 in column x:
0     True
1    False
2    False
3    False
Name: x, dtype: bool

Elements < 5 in column y:
0     True
1    False
2    False
3     True
Name: y, dtype: bool

Elements != 5 in column y:
0     True
1     True
2    False
3     True
Name: y, dtype: bool

Filtering Rows with Complex Criteria

You can use the boolean results to filter rows that meet specific conditions ?

import pandas as pd

df = pd.DataFrame(
    {
        "x": [5, 2, 7, 0],
        "y": [4, 7, 5, 1],
        "z": [9, 3, 5, 1]
    }
)

# Filter rows where x > 2
print("Rows where x > 2:")
print(df[df['x'] > 2])

# Multiple conditions using & (and) operator
print("\nRows where x > 2 AND y < 6:")
print(df[(df['x'] > 2) & (df['y'] < 6)])

# Multiple conditions using | (or) operator
print("\nRows where x == 0 OR z > 8:")
print(df[(df['x'] == 0) | (df['z'] > 8)])

Rows where x > 2:
   x  y  z
0  5  4  9
2  7  5  5

Rows where x > 2 AND y < 6:
   x  y  z
0  5  4  9
2  7  5  5

Rows where x == 0 OR z > 8:
   x  y  z
0  5  4  9
3  0  1  1

Common Comparison Operations

Conclusion

Pandas comparison operations return boolean Series that can be used for filtering DataFrames. Use parentheses around conditions when combining with & or | operators for complex criteria selection.