Smooth approximation of the hinge loss function I came across a paper but the smooth approximation for the hinge loss function is wrong. Can someone guide me to the proper smooth approximation (using polynomials) of the function $$h(x)=\max(0,1-x)$$ which is exact when $|x| \ge \rho$, where $\rho$ can be made arbitrarily small?
 A: Here are the details for those who might feel too lazy to chase the links in the comments above.
The Hinge loss for $x \in R$ is defined as
$$H(x) = \max(0, 1-x)$$
Jason Rennie in his article "Smooth Hinge Classification" describes the following smooth version of the Hinge loss (a smoothed version was being sought because of discontinuity in the derivative at $x=1$).
Rennie defines (the definition seems natural enough that somebody might have also found a similar one; I will be happy to be corrected) the smoothed Hinge loss:
$$H_s(x) = \begin{cases} \tfrac{1}{2}-x & x \le 0,\\\\
\tfrac{1}{2}(1-x)^2 & 0 < x < 1\\\\
0 & x \ge 1
\end{cases}$$
This loss is smooth, and its derivative is continuous (verified trivially).
Rennie goes on to discuss a parametrized family of smooth Hinge-losses $H_s(x; \alpha)$. Additionally, several other variations are possible, depending on what numerical behavior seems more appropriate for an application.
A: The paper Differentially private empirical risk minimization by K. Chaudhuri, C. Monteleoni, A. Sarwate (Journal of Machine Learning Research 12 (2011) 1069-1109), gives two alternatives of "smoothed" hinge loss which are doubly differentiable.
The paper is linked here (pdf); the approximations are given p.14-15 (p.1082-1083 in journal); see the extract 1 and 
extract 2.
A: I realize this is an old question, but it seems worth noting that the requested hinge function is probably the Huber loss. This is different from the ones discussed in other answers, and actually has a parameter $\rho \geq 0$ that can be made arbitrarily small. It can be defined as
$$
h(x ; \rho) = \begin{cases}
1 - x & \text{ if } x \leq 1 - \rho \\
\frac{1}{4\rho} (1 - x + \rho)^2 & \text{ if } 1 - \rho < x \leq 1 + \rho \\
0 & \text{ if } x > 1 + \rho
\end{cases}
$$
As in the other answers, this is not "smooth" in the infinitely-differentiable sense, but it is in $C^1$. It is straightforward to show that $\lim_{\rho \rightarrow 0} h(x ; \rho) = \max(1 - x, 0)$ and it is exact for $|x - 1| \geq \rho$. This hinge function is sometimes used in SVMs [e.g., 1,2]. Here's a quick plot:

