How to Use Logistic Regression for Regression

Logistic regression is a type of classification algorithm used in machine learning very frequently. It is one of the easiest and most efficient classification algorithms, which is tried out on almost every model built for classification problems. However, we can also use logistic regression for solving regression problems, although in such cases, we cannot expect great accuracy and reliability of the model.

In this article, we will discuss logistic regression, how we can use it for regression problems, what steps are needed, and the code example for the same. This article will help one to understand the behavior of logistic regression for regression problems and will be able to answer interview questions related to the same.

About Logistic Regression

Logistic regression is a type of machine learning algorithm that works on the principle of perceptron trick code. Here similar to linear regression, a straight line or a line of regression is drawn, which basically separates the categories or the classes of the target variable, which is the categorical variable.

Although we call it a logistic regression instead of classification because it works almost the same as the linear regression algorithms, and hence it's called regression instead classification. The key difference between linear regression and logistic regression is the only sigmoid function in logistic regression is used to categorize the observations and give the output at the end.

Hence there is no harm in calling logistic regression a regression as the working mechanism of the algorithm is almost the same as linear regression, and the line is obtained the same as linear regression.

Using Logistic Regression for Regression

As we know that logistic regression works almost the same as linear regression, it just transforms the probabilities of the categories between 0 to 1 using the sigmoid function. But we can also use logistic regression for regression problems as, in the end, it works on the line like linear regression only.

Although here, the performance of the model will not be as good as other well-known regression algorithms, and the model would not be that reliable, we can still try experimenting with the logistic regression.

There are basically two main requirements for using logistic regression for a regression problem.

Modify the Output

As we know that logistic regression uses a sigmoid function as the logistic function, whichh transforms the values of the output between 0 to 1 to get an idea about the probability of the binary classes. Here to use it as a regression model, we can transform the output to be a continuous value by multiplying it with any type of scaling factor.

So basically, we cannot use the sigmoid function directly to get an output; we have to modify the output and make it a continuous variable instead of a categorical variable.

Change the Loss Function

Now as we are shifted to the regression problem, we have to use a regression-based loss function instead of a classification loss function like accuracy score, precision score, recall scope, etc. We can use the r2 score, mean squared error, RMSE score, etc., in such cases.

Note that by not changing the loss function of the model, you will get very low or minus accuracy or the classification scores, as the model has been converted to the regression model, and the classification loss function is being used.

Code Example

Now as we know that we can not use a classification algorithm to solve regression problems, and hence logistic regression can not be used for regression problems technically as it is designed for classification problems only. Still, if we want, then we can bin the continuous variables here in order to make them look like categories.

Basically, we can take the continuous variables and divide them into a bin where the values of the particular continuous variables will be considered as a single bin, and that will be a single individual category for the logistic regression.

Now these values will be provided to the logistic regression model, and the model will perform on the same. However, it is most recommended to do so as there are lots of regression algorithms that will perform way better than logistic regression on regression problems.

Here is the code example where we have a continuous variable as the target variable and which is converted to the categorical variable by performing the binning operation to the same.

Example

import numpy as np
from sklearn.linear_model import LogisticRegression

# Create a dummy dataset with one feature and a target variable
X = np.array([1, 2, 3, 4, 5, 6, 7, 8])
y = np.array([10, 15, 20, 25, 30, 35, 40, 45])

# Bin the target variable into multiple categories
bins = np.array([0, 20, 40, 60])
y_cat = np.digitize(y, bins)

# Create a logistic regression object and fit the data
lr = LogisticRegression()
lr.fit(X.reshape(-1, 1), y_cat)

# Predict the target variable for new data points
X_new = np.array([9, 10]).reshape(-1, 1)
y_pred = lr.predict(X_new)
 
# Convert the predicted categories back into continuous values
y_pred_continuous = (np.take(bins, y_pred) + np.take(bins, y_pred - 1))/2
print(y_pred_continuous)

Output

[50. 50.]

Key Takeaways

• Logistic regression is a type of classification algorithm which is majorly used in binary classification problems.

• It is called logistic regression, as the working of the algorithm is almost similar to linear regression.

• We can use logistic regression as a regression model by modifying the outputs and the loss function of the model.

• We can convert the outputs of the model by performing binning or binarization operations on the target variable.

• If we want to use logistic regression for regression problems, then we can use loss functions like r2 score, mean squared error, and RMSE score.

Conclusion

In this article, we discuss about logistic regression, how we can use it as a regression model, what steps are needed, and how we can perform the same with a code example. This article will help one to understand the working of logistic regression on regression problems and will able to understand and analyze the performance of the same on regression datasets.