Show Sidebar Hide Sidebar

Lasso and Elastic Net in Scikit-learn

Lasso and elastic net (L1 and L2 penalisation) implemented using a coordinate descent.

The coefficients can be forced to be positive.

New to Plotly?

Plotly's Python library is free and open source! Get started by downloading the client and reading the primer.
You can set up Plotly to work in online or offline mode, or in jupyter notebooks.
We also have a quick-reference cheatsheet (new!) to help you get started!

Version

In [1]:
import sklearn
sklearn.__version__
Out[1]:
'0.18.1'

Imports

In [2]:
print(__doc__)

import plotly.plotly as py
import plotly.graph_objs as go

from itertools import cycle
import numpy as np

from sklearn.linear_model import lasso_path, enet_path
from sklearn import datasets
Automatically created module for IPython interactive environment

Calculations

In [3]:
diabetes = datasets.load_diabetes()
X = diabetes.data
y = diabetes.target

X /= X.std(axis=0)  # Standardize data (easier to set the l1_ratio parameter)

# Compute paths

eps = 5e-3  # the smaller it is the longer is the path

print("Computing regularization path using the lasso...")
alphas_lasso, coefs_lasso, _ = lasso_path(X, y, eps, fit_intercept=False)

print("Computing regularization path using the positive lasso...")
alphas_positive_lasso, coefs_positive_lasso, _ = lasso_path(
    X, y, eps, positive=True, fit_intercept=False)
print("Computing regularization path using the elastic net...")
alphas_enet, coefs_enet, _ = enet_path(
    X, y, eps=eps, l1_ratio=0.8, fit_intercept=False)

print("Computing regularization path using the positive elastic net...")
alphas_positive_enet, coefs_positive_enet, _ = enet_path(
    X, y, eps=eps, l1_ratio=0.8, positive=True, fit_intercept=False)

colors = 10 * ['blue', 'red', 'green', 'cyan', 'black']
Computing regularization path using the lasso...
Computing regularization path using the positive lasso...
Computing regularization path using the elastic net...
Computing regularization path using the positive elastic net...

Lasso and Elastic-Net Paths

In [4]:
neg_log_alphas_lasso = -np.log10(alphas_lasso)
neg_log_alphas_enet = -np.log10(alphas_enet)
data = []
i=1
for coef_l, coef_e, c in zip(coefs_lasso, coefs_enet, colors):
    if(i==1):
        leg=True
    else:
        leg=False
        
    l1 = go.Scatter(x=neg_log_alphas_lasso, y=coef_l, 
                    mode='lines', line=dict(color=c, width=1),
                    showlegend=leg,
                    name='Lasso')
    l2 = go.Scatter(x=neg_log_alphas_enet, y=coef_e, 
                    mode='lines', 
                    line=dict(color=c, dash='dash', width=1),
                    showlegend=leg,
                    name='Elastic-Net')
    data.append(l1)
    data.append(l2)
    i+=1

layout = go.Layout(title='Lasso and Elastic-Net Paths',
                   hovermode='closest',
                   xaxis=dict(title='-Log(alpha)'),
                   yaxis=dict(title='coefficients'))
fig = go.Figure(data=data, layout=layout)
In [5]:
py.iplot(fig)
Out[5]:

Lasso and positive Lasso

In [6]:
neg_log_alphas_positive_lasso = -np.log10(alphas_positive_lasso)
data = []
i=1

for coef_l, coef_pl, c in zip(coefs_lasso, coefs_positive_lasso, colors):
    if(i==1):
        leg=True
    else:
        leg=False
        
    l1 = go.Scatter(x=neg_log_alphas_lasso, y=coef_l, 
                    mode='lines', line=dict(color=c, width=1),
                    showlegend=leg,
                    name='Lasso')
    l2 = go.Scatter(x=neg_log_alphas_positive_lasso, y=coef_pl, 
                    mode='lines', 
                    line=dict(color=c, dash='dash', width=1),
                    showlegend=leg,
                    name='Elastic-Net')
    data.append(l1)
    data.append(l2)
    i+=1
    
layout = go.Layout(title='Lasso and positive Lasso',
                   hovermode='closest',
                   xaxis=dict(title='-Log(alpha)'),
                   yaxis=dict(title='coefficients'))
fig = go.Figure(data=data, layout=layout)
In [7]:
py.iplot(fig)
Out[7]:

Elastic-Net and Positive Elastic-Net

In [8]:
neg_log_alphas_positive_enet = -np.log10(alphas_positive_enet)
data = []
i=1

for (coef_e, coef_pe, c) in zip(coefs_enet, coefs_positive_enet, colors):
    if(i==1):
        leg=True
    else:
        leg=False
        
    l1 = go.Scatter(x=neg_log_alphas_enet, y=coef_e, 
                    mode='lines', line=dict(color=c, width=1),
                    showlegend=leg,
                    name='Lasso')
    l2 = go.Scatter(x=neg_log_alphas_positive_enet, y=coef_pe, 
                    mode='lines', 
                    line=dict(color=c, dash='dash', width=1),
                    showlegend=leg,
                    name='Elastic-Net')
    data.append(l1)
    data.append(l2)
    i+=1
    
layout = go.Layout(title='Elastic-Net and positive Elastic-Net',
                   hovermode='closest',
                   xaxis=dict(title='-Log(alpha)'),
                   yaxis=dict(title='coefficients'))
fig = go.Figure(data=data, layout=layout)
In [9]:
py.iplot(fig)
Out[9]:

License

Author:

    Alexandre Gramfort <alexandre.gramfort@inria.fr>

License:

    BSD 3 clause
Still need help?
Contact Us

For guaranteed 24 hour response turnarounds, upgrade to a Developer Support Plan.