Compute Classification Report and Confusion Matrics in Python

In machine learning, classification problems require careful evaluation to understand model performance. The classification report and confusion matrix are essential tools that help us evaluate classification models and identify where they make mistakes.

This article will explore these evaluation methods through practical Python examples, covering their components, interpretation, and implementation using scikit-learn.

What is a Confusion Matrix?

A confusion matrix is a table that summarizes the performance of a classification model by comparing predicted vs. actual values. It contains four key components ?

• True Positive (TP): Model correctly predicts positive class
• True Negative (TN): Model correctly predicts negative class
• False Positive (FP): Model incorrectly predicts positive (Type I error)
• False Negative (FN): Model incorrectly predicts negative (Type II error)

Key Performance Metrics

From the confusion matrix, we can calculate several important metrics ?

Accuracy

Measures overall correctness of the model:

Accuracy = (TP + TN) / (TP + TN + FP + FN)

Precision

Measures accuracy of positive predictions:

Precision = TP / (TP + FP)

Recall (Sensitivity)

Measures ability to find all positive instances:

Recall = TP / (TP + FN)

F1-Score

Harmonic mean of precision and recall:

F1-Score = 2 × (Precision × Recall) / (Precision + Recall)

What is a Classification Report?

A classification report provides a comprehensive summary of model performance, including precision, recall, F1-score, and support for each class. Support represents the number of actual occurrences of each class in the dataset.

Computing Classification Report and Confusion Matrix

Let's create a practical example using scikit-learn to compute both metrics ?

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import classification_report, confusion_matrix, accuracy_score

# Generate sample dataset
np.random.seed(42)
X = np.random.rand(200, 5)
y = np.random.randint(2, size=200)

# Split dataset
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# Train model
model = LogisticRegression()
model.fit(X_train, y_train)

# Make predictions
y_pred = model.predict(X_test)

# Calculate metrics
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.3f}")

Accuracy: 0.517

Now let's generate the classification report ?

# Classification Report
report = classification_report(y_test, y_pred)
print("Classification Report:")
print(report)

Classification Report:
              precision    recall  f1-score   support

           0       0.53      0.56      0.55        27
           1       0.50      0.48      0.49        33

    accuracy                           0.52        60
   macro avg       0.52      0.52      0.52        60
weighted avg       0.51      0.52      0.51        60

Finally, let's compute the confusion matrix ?

# Confusion Matrix
conf_matrix = confusion_matrix(y_test, y_pred)
print("Confusion Matrix:")
print(conf_matrix)

Confusion Matrix:
[[15 12]
 [17 16]]

Interpreting the Results

From our confusion matrix [[15 12] [17 16]]:

• True Positives (Class 1): 16
• True Negatives (Class 0): 15
• False Positives: 17
• False Negatives: 12

The classification report shows balanced performance across both classes, with macro and weighted averages providing different perspectives on overall performance.

Enhanced Visualization

For better interpretation, we can visualize the confusion matrix ?

import matplotlib.pyplot as plt
import seaborn as sns

# Create heatmap
plt.figure(figsize=(6, 4))
sns.heatmap(conf_matrix, annot=True, fmt='d', cmap='Blues', 
            xticklabels=['Class 0', 'Class 1'], 
            yticklabels=['Class 0', 'Class 1'])
plt.title('Confusion Matrix')
plt.ylabel('Actual')
plt.xlabel('Predicted')
plt.show()

Conclusion

Classification reports and confusion matrices are essential tools for evaluating machine learning models. They provide detailed insights into model performance across different classes and help identify specific areas for improvement. Use these metrics together to get a comprehensive understanding of your classification model's strengths and weaknesses.