MathOverflow is a question and answer site for professional mathematicians. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I came across this paper but the smooth approximation for the hinge loss function is wrong. Can someone guide me to the proper smooth approximation (using polynomials) of the function $h(x)=max(0,1-x)$ which is exact when $|x|>=\rho$, where $\rho$ can be made arbitrarily small?


share|cite|improve this question
You might as well work with $max(0,-x)$, which makes your bound involving $\rho$ make sense. And then you can work with a fixed $\rho$ ( $= 1$, say) – David Roberts Jan 7 '11 at 5:35
But I don't know if you can do it with polynomials. – David Roberts Jan 7 '11 at 5:35
The paper that you have linked works only for UIUC; please fix the link so that others can see what paper you are referring to. – Suvrit Jan 7 '11 at 10:03
Also, what about the approximation in: – Suvrit Jan 7 '11 at 10:06
Sorry about the link. The paper can be viewed here: Actually, I found the proper coefficients for the order 4 polynomial. The paper has a typo on page 5. Instead of $\frac{3}{2}x$ it should be $\frac{1}{2}x$. The link for the smooth quadratic hinge function was a great help. Thanks Suvrit. – Bernard Jan 7 '11 at 11:59

Here are the details for those who might feel too lazy to chase the links in the comments above.

The Hinge loss for $x \in R$ is defined as $$H(x) = \max(0, 1-x)$$

Jason Rennie in his article "Smooth Hinge Classification" describes the following smooth version of the Hinge loss (a smoothed version was being sought because of discontinuity in the derivative at $x=1$).

Rennie defines (the definition seems natural enough that somebody might have also found a similar one; I will be happy to be corrected) the smoothed Hinge loss: $$H_s(x) = \begin{cases} \tfrac{1}{2}-x & x \le 0,\\\\ \tfrac{1}{2}(1-x)^2 & 0 < x < 1\\\\ 0 & x \ge 1 \end{cases}$$

This loss is smooth, and its derivative is continuous (verified trivially).

Rennie goes on to discuss a parametrized family of smooth Hinge-losses $H_s(x; \alpha)$. Additionally, several other variations are possible, depending on what numerical behavior seems more appropriate for an application.

share|cite|improve this answer
Note that $H_s$ is $C^1$, not smooth in the usual sense (infinitely differentiable) (this is for random students wandering by, in case they are confused) – David Roberts Feb 4 '11 at 22:34

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.