Scikit Learn - MultiTaskElasticNet

MultiTaskElasticNet

It is an Elastic-Net model that allows to fit multiple regression problems jointly enforcing the selected features to be same for all the regression problems, also called tasks. Sklearn provides a linear model named MultiTaskElasticNet, trained with a mixed L1, L2-norm and L2 for regularisation, which estimates sparse coefficients for multiple regression problems jointly. In this, the response y is a 2D array of shape (n_samples, n_tasks).

Following is the objective function to minimize −

$$\displaystyle\min\limits_{W}\frac{1}{2n_{samples}}\lVert X_{W}-Y\rVert_{fro}^2+\alpha\rho\lVert W\rVert_{21}+\frac{\alpha\lgroup 1-\rho\rgroup}{2}\ \lVert W\rVert_{fro}^2$$

As in MultiTaskLasso, here also, Fro indicates the Frobenius norm −

$$\lVert A\rVert_{Fro}=\sqrt{\displaystyle\sum\limits_{ij}}a_{ij}^2$$

And L1L2 leads to the following −

$$\lVert A\rVert_{21}=\displaystyle\sum\limits_{i} \sqrt{\displaystyle\sum\limits_{j}}a_{ij}^2$$

The parameters and the attributes for MultiTaskElasticNet are like that of ElasticNet. The only difference is in li_ratio i.e. ElasticNet mixing parameter. In MultiTaskElasticNet its range is 0 < l1_ratio < = 1. If l1_ratio = 1, the penalty would be L1/L2 penalty. If l1_ratio = 0, the penalty would be an L2 penalty. If the value of l1 ratio is between 0 and 1, the penalty would be the combination of L1/L2 and L2.

And, opposite to ElasticNet, MultiTaskElasticNet doesn’t have precompute attribute.

Implementation Example

To show the difference, we are implementing the same example as we did in Multi-task Lasso −

from sklearn import linear_model
MTENReg = linear_model.MultiTaskElasticNet(alpha = 0.5)
MTENReg.fit([[0,0], [1, 1], [2, 2]], [[0, 0],[1,1],[2,2]])

Output

MultiTaskElasticNet(alpha = 0.5, copy_X = True, fit_intercept = True, l1_ratio = 0.5,
max_iter = 1000, normalize = False, random_state = None,
selection = 'cyclic', tol = 0.0001, warm_start = False)

Example

#Predicting new values
MTENReg.predict([[1,0]])

Output

array([[0.69056563, 0.69056563]])

Example

#weight vectors
MTENReg.coef_

Output

array([[0.30943437, 0.30938224],
[0.30943437, 0.30938224]])

Example

#Calculating intercept
MTENReg.intercept_

Output

array([0.38118338, 0.38118338])

Example

#Calculating number of iterations
MTENReg.n_iter_

Output