# smooth approximation of the hinge loss function

I came across this paper but the smooth approximation for the hinge loss function is wrong. Can someone guide me to the proper smooth approximation (using polynomials) of the function $h(x)=max(0,1-x)$ which is exact when $|x|>=\rho$, where $\rho$ can be made arbitrarily small?

Thanks

-
You might as well work with $max(0,-x)$, which makes your bound involving $\rho$ make sense. And then you can work with a fixed $\rho$ ( $= 1$, say) – David Roberts Jan 7 '11 at 5:35
But I don't know if you can do it with polynomials. – David Roberts Jan 7 '11 at 5:35
The paper that you have linked works only for UIUC; please fix the link so that others can see what paper you are referring to. – Suvrit Jan 7 '11 at 10:03
Also, what about the approximation in: people.csail.mit.edu/jrennie/writing/smoothHinge.pdf – Suvrit Jan 7 '11 at 10:06
Sorry about the link. The paper can be viewed here: cs.uiuc.edu/homes/afarhad2/index_files/scene_discovery.pdf. Actually, I found the proper coefficients for the order 4 polynomial. The paper has a typo on page 5. Instead of $\frac{3}{2}x$ it should be $\frac{1}{2}x$. The link for the smooth quadratic hinge function was a great help. Thanks Suvrit. – Bernard Jan 7 '11 at 11:59

Here are the details for those who might feel too lazy to chase the links in the comments above.

The Hinge loss for $x \in R$ is defined as $$H(x) = \max(0, 1-x)$$

Jason Rennie in his article "Smooth Hinge Classification" describes the following smooth version of the Hinge loss (a smoothed version was being sought because of discontinuity in the derivative at $x=1$).

Rennie defines (the definition seems natural enough that somebody might have also found a similar one; I will be happy to be corrected) the smoothed Hinge loss: $$H_s(x) = \begin{cases} \tfrac{1}{2}-x & x \le 0,\\\\ \tfrac{1}{2}(1-x)^2 & 0 < x < 1\\\\ 0 & x \ge 1 \end{cases}$$

This loss is smooth, and its derivative is continuous (verified trivially).

Rennie goes on to discuss a parametrized family of smooth Hinge-losses $H_s(x; \alpha)$. Additionally, several other variations are possible, depending on what numerical behavior seems more appropriate for an application.

-
Note that $H_s$ is $C^1$, not smooth in the usual sense (infinitely differentiable) (this is for random students wandering by, in case they are confused) – David Roberts Feb 4 '11 at 22:34