Show Sidebar Hide Sidebar

# SGD Convex Loss Functions in Scikit-learn

A plot that compares the various convex loss functions supported by sklearn.linear_model.SGDClassifier

#### New to Plotly?¶

You can set up Plotly to work in online or offline mode, or in jupyter notebooks.
We also have a quick-reference cheatsheet (new!) to help you get started!

### Version¶

In [1]:
import sklearn
sklearn.__version__
Out[1]:
'0.18.1'

### Imports¶

In [2]:
print(__doc__)

import plotly.plotly as py
import plotly.graph_objs as go

import numpy as np
Automatically created module for IPython interactive environment

### Calculations¶

In [3]:
def modified_huber_loss(y_true, y_pred):
z = y_pred * y_true
loss = -4 * z
loss[z >= -1] = (1 - z[z >= -1]) ** 2
loss[z >= 1.] = 0
return loss

xmin, xmax = -4, 4
xx = np.linspace(xmin, xmax, 100)
lw = 2

### Plot Results¶

In [4]:
trace1 = go.Scatter(x=[xmin, 0, 0, xmax],
y=[1, 1, 0, 0],
mode='lines',
line=dict(color='gold', width=lw),
name="Zero-one loss")

trace2 = go.Scatter(x=xx,
y=np.where(xx < 1, 1 - xx, 0),
mode='lines',
line=dict(color='teal', width=lw),
name="Hinge loss")

trace3 = go.Scatter(x=xx,
y=-np.minimum(xx, 0),
mode='lines',
line=dict(color='yellowgreen', width=lw),
name="Perceptron loss")

trace4 = go.Scatter(x=xx,
y=np.log2(1 + np.exp(-xx)),
mode='lines',
line=dict(color='cornflowerblue', width=lw),
name="Log loss")

trace5 = go.Scatter(x=xx,
y=np.where(xx < 1, 1 - xx, 0) ** 2,
mode='lines',
line=dict(color='orange', width=lw),
name="Squared hinge loss")

trace6 = go.Scatter(x=xx,
y=modified_huber_loss(xx, 1),
mode='lines',
line=dict(color='darkorchid', width=lw, dash='dash'),
name="Modified Huber loss")

data = [trace1, trace2, trace3, trace4, trace5, trace6]

layout = go.Layout(xaxis=dict(title=r"Decision function <i>f(x)</i>",
zeroline=False, showgrid=False),
yaxis=dict(title="<i>L(y, f(x))</i>", range=[0, 8],
showgrid=False),
hovermode='closest')

fig = go.Figure(data=data, layout=layout)
In [5]:
py.iplot(fig)
Out[5]:
Still need help?