Let $x_1,\ldots,x_n \in \mathbb R^d$ and $y_1,\ldots,y_n \in \{\pm 1\}$, and $\epsilon, h \gt 0$. Define $\theta(t) := Q((t-\epsilon)/h)$, where $Q(z) := \int_{z}^\infty \phi (z)\mathrm{d}z$ is the Gaussian survival function. Define $E:\mathbb R^d \to \mathbb R$ by
$$ E(w) := \frac{1}{n}\sum_{i=1}^n \theta(m_i(w)), $$
where $m_i(w) := y_i x_i^\top w$ defines the margin at the labeled data point $(x_i,y_i)$.
Question. Is there a convergent (sub)gradient-type scheme which can be used to optimize $L$ over the unit-sphere in $\mathbb R^d$ ?
Some observations
Notice that since each $m_i$ is differentiable on $\mathbb R^d$ with gradient $\nabla m_i(w) = y_ix_i$, the function $E$ is infinitely-differentiable on $\mathbb R^d$ with gradient and hessian given respectively by $$ \begin{split} \nabla E(w) &= -\dfrac{1}{nh}\sum_{i=1}^n \phi (( m_i(w)-\epsilon )/h)y_ix_i,\\ \nabla^2 E(w) &= \frac{1}{nh^2}\sum_{i=1}^n \phi '(( m_i(w)-\epsilon )/h) x_ix_i^\top = \frac{1}{nh^2}X^\top D(w) X, \end{split} $$ where $D(w)$ is an $n \times n$ diagonal matrix with $D(w)_{ii} = \phi '(( m_i(w)-\epsilon )/h) $.
In particular, note that $E$ is $L$-smooth with $L=O(\|X\|_{op}^2/(nh^2))$, since $$ \frac{nh^2}{\|X\|_{op}^2} \cdot \sup_{w \in \mathbb R^d} \|\nabla^2 E(w)\|_{op} \le \sup_{w \in \mathbb R^d} \max_{1 \le i \le n}|D(w)_{ii}| \le \|\phi'\|_\infty = \frac{1}{\sqrt{2\pi e}}. $$
