Let $x_1,\ldots,x_n \in \mathbb R^d$ and $y_1,\ldots,y_n \in \{\pm 1\}$, and $\epsilon, h \gt 0$. Define $\theta(t) := Q((t-\epsilon)/h)$, where $Q(z) := \int_{z}^\infty \phi_h(z)\mathrm{d}z$ is the survival function of the Gaussian distribution $N(0,h^2)$ with density function $\phi_h$. Define $E:\mathbb R^d \to \mathbb R$ by

$$ E(w) := \frac{1}{n}\sum_{i=1}^n \theta(m_i(w)), $$

where $m_i(w) := y_i x_i^\top w$ defines the margin at the labeled data point $(x_i,y_i)$.

Question. Is there a convergent (sub)gradient-type scheme which can be used to optimize $L$ over the unit-sphere in $\mathbb R^d$ ?

Notice that since each $m_i$ is differentiable on $\mathbb R^d$ with gradient $\nabla m_i(w) = -y_ix_i$, the function $E$ is infinitely-differentiable on $\mathbb R^d$ with gradient and hessian given respectively by $$ \begin{split} \nabla E(w) &= -\dfrac{1}{nh}\sum_{i=1}^n \phi (( m_i(w)-\epsilon )/h)y_ix_i,\\ \nabla^2 L(w) &= \frac{1}{nh^2}\sum_{i=1}^n \phi '(( m_i(w)-\epsilon )/h) x_ix_i^\top = \frac{1}{nh^2}X^\top D(w) X, \end{split} $$ where $D(w)$ is an $n \times n$ diagonal matrix with $D(w)_{ii} = \phi '(( m_i(w)-\epsilon )/h) $.

In particular, note that $E$ is $L$-smooth with $$ L = \sup_{w \in \mathbb R^d} \|\nabla^2 E(w)\|_{op} \le \|X\|_{op}^2\cdot\sup_{w \in \mathbb R^d} \max_{1 \le i \le n}|\phi '(( m_i(w)-\epsilon )/h)| \le \frac{1}{\sqrt{2\pi e}}\|X\|_{op}^2. $$