Python - Implementation of Polynomial Regression

Polynomial Regression is a form of linear regression in which the relationship between the independent variable x and dependent variable y is modeled as an nth degree polynomial. Polynomial regression fits a nonlinear relationship between the value of x and the corresponding conditional mean of y, denoted E(y |x).

Unlike simple linear regression that creates a straight line, polynomial regression can capture curved relationships in data by using polynomial terms like x², x³, etc.

Creating Sample Data

First, let's create some sample data to demonstrate polynomial regression ?

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures

# Create sample data
np.random.seed(42)
temperature = np.array([20, 30, 40, 50, 60, 70, 80, 90, 100, 110]).reshape(-1, 1)
pressure = 2 * temperature.flatten()**2 + 10 * temperature.flatten() + np.random.normal(0, 100, 10)

# Create DataFrame
data = pd.DataFrame({'Temperature': temperature.flatten(), 'Pressure': pressure})
print(data)

   Temperature      Pressure
0           20    968.967143
1           30   2052.272303
2           40   3564.813504
3           50   5372.332739
4           60   7575.007382
5           70  10134.358680
6           80  13014.192442
7           90  16326.896962
8          100  19944.434321
9          110  24095.344203

Linear vs Polynomial Regression

Linear Regression

# Prepare data
X = data[['Temperature']]
y = data['Pressure']

# Fit linear regression
linear_reg = LinearRegression()
linear_reg.fit(X, y)

# Make predictions
linear_pred = linear_reg.predict(X)

print("Linear Regression Score:", linear_reg.score(X, y))

Linear Regression Score: 0.9814451218370679

Polynomial Regression

# Create polynomial features
poly_features = PolynomialFeatures(degree=2)
X_poly = poly_features.fit_transform(X)

# Fit polynomial regression
poly_reg = LinearRegression()
poly_reg.fit(X_poly, y)

# Make predictions
poly_pred = poly_reg.predict(X_poly)

print("Polynomial Regression Score:", poly_reg.score(X_poly, y))
print("Polynomial features shape:", X_poly.shape)

Polynomial Regression Score: 0.9999847710386378
Polynomial Regression Score: (10, 3)

Visualizing Results

# Create plots
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))

# Linear regression plot
ax1.scatter(X, y, color='blue', alpha=0.7)
ax1.plot(X, linear_pred, color='red', linewidth=2)
ax1.set_title('Linear Regression')
ax1.set_xlabel('Temperature')
ax1.set_ylabel('Pressure')

# Polynomial regression plot
ax2.scatter(X, y, color='blue', alpha=0.7)
ax2.plot(X, poly_pred, color='red', linewidth=2)
ax2.set_title('Polynomial Regression (degree=2)')
ax2.set_xlabel('Temperature')
ax2.set_ylabel('Pressure')

plt.tight_layout()
plt.show()

Making Predictions

# Predict for new temperature value
new_temp = np.array([[115]])

# Linear regression prediction
linear_prediction = linear_reg.predict(new_temp)
print(f"Linear regression prediction for 115°: {linear_prediction[0]:.2f}")

# Polynomial regression prediction
new_temp_poly = poly_features.transform(new_temp)
poly_prediction = poly_reg.predict(new_temp_poly)
print(f"Polynomial regression prediction for 115°: {poly_prediction[0]:.2f}")

Linear regression prediction for 115°: 25183.28
Polynomial regression prediction for 115°: 27599.65

Comparison

Conclusion

Polynomial regression extends linear regression to model curved relationships by using polynomial features. It provides better fit for nonlinear data but requires careful selection of polynomial degree to avoid overfitting.