Convergence of estimator given by a fixed point Let $X$ be a non-negative random variable with cdf $F$ and define
$$G(s) = E[\max(0,u(X)-sX)],$$ where $u$ is some real function.
Let $s_0$ be the unique fixed point of $G$.
Now let $X_1,\dots,X_t$ be a sequence of samples drawn independantly from $F$.
Let
$$G_t(s) = \frac{1}{t}\sum_{i=1}^t \max(0,u(X_i)-sX_i)$$
and let $s_t$ be the unique fixed point of $G_t$.
Is it true that $s_t$ converges almost surely to $s_0$, and is it possible to say something about the speed of convergence? If not, how could I construct a converging estimator of $s_0$?
 A: $\newcommand\N{\{1,2,\dots\}}
\newcommand\NN{\{1,2,\dots,\infty\}}
\newcommand\si{\sigma}
\newcommand{\ep}{\varepsilon}$
Suppose that $E u(X)_+<\infty$, where $x_+:=\max(0,x)$. Then the convergence takes place, with the rate $O(1/\sqrt t)$ (as $t\to\infty$) if we also assume that $EX<\infty$, $E u(X)_+^a<\infty$ for some $a>2$, and
$$\si(s):=\sqrt{Var\,(u(X)-s X)_+}\ne0$$ 
for any real $s$. 
Indeed, let 
$$G_\infty(s):=G(s)=E(u(X)-sX)_+,$$
and then let 
$$H_t(s):=G_t(s)-s$$
for all $t\in\NN$, so that $EH_t(s)=H_\infty(s)$. For each $t$, the function $G_t$ is nonincreasing and hence $H_t$ is strictly decreasing. Moreover, by dominated convergence, $G_\infty$ is continuous and hence $H_\infty$ is continuous. Also, $H_t$ is obviously continuous for $t\in\N$. Further, $H_t(0)\ge0>-\infty=H_t(\infty-)$. So, for each $t\in\NN$, the function $H_t$ has a unique zero in $[0,\infty)$, that is, the function $G_t$ has a unique fixed point, say $s_t$, in $[0,\infty)$.
Also, using the condition $E u(X)_+^a<\infty$ with $a>2$ and, again, the dominated convergence theorem, we see that, for each $b\in[0,a]$, $E(u(X)-sX)_+^b$ is continuous in $s$ and hence bounded in $s$ in any compact interval. 
So, by the Lyapunov--Bikelis inequality (see e.g. inequality (1)), 
\begin{equation}
 P(Z_t(s)>z)-P(Z>z)\to0\tag{1}
\end{equation}
uniformly over all real $z$ and all $s$ in any compact interval, where $Z\sim N(0,1)$ and 
$$Z_t(s):=\frac{H_t(s)-H_\infty(s)}{\si(s)/\sqrt t}.$$
So, for any real $z$ and any $t\in\N$
\begin{align*}
P(s_t>s_\infty+z/\sqrt t)&=P(H_t(s_t)<H_t(s_\infty+z/\sqrt t)) \\
&=P(H_t(s_\infty+z/\sqrt t)>0) \\
& =P\Big(Z_t(s_\infty+z/\sqrt t)>\frac{-H_\infty(s_\infty+z/\sqrt t)}{\si(s_\infty+z/\sqrt t)/\sqrt t}\Big) \\ 
& =P\Big(Z>\frac{-H_\infty(s_\infty+z/\sqrt t)}{\si(s_\infty+z/\sqrt t)/\sqrt t}\Big)+o(1),   
\end{align*}
by (1). 
Next, again by dominated convergence, the right derivative of $H_\infty$ at any real $s$ is $-\mu^+(s)-1$, where 
$$\mu^+(s):=EX1_{u(X)-sX>0}.$$
So, for any real $z>0$, $H_\infty(s_\infty+z/\sqrt t)=H_\infty(s_\infty+z/\sqrt t)-H_\infty(s_\infty)\sim-(1+\mu^+(s_\infty))z/\sqrt t$. 
Also, using again the mentioned continuity of $E(u(X)-sX)_+^b$ in $s$ for each $b\in[0,a]$, we see that $\si(s_\infty+z/\sqrt t)\to\si(s_\infty)$. 
We conclude that for any real $z>0$
\begin{align*}
P(s_t-s_\infty>z/\sqrt t)&\to P\Big(Z>\frac{(1+\mu^+(s_\infty))z}{\si(s_\infty)}\Big).   
\end{align*}
Similarly, for any real $z>0$
\begin{align*}
P(s_t-s_\infty<-z/\sqrt t)&\to P\Big(Z<-\frac{(1+\mu^-(s_\infty))z}{\si(s_\infty)}\Big), 
\end{align*}
where $\mu^-(s):=EX1_{u(X)-sX\ge0}$. 
Thus indeed, $s_t$ converges to $s_\infty$ at the rate $O(1/\sqrt t)$. 

Let us also show that $s_t\to s_\infty$ almost surely (a.s). Take any real $\ep>0$. Since the function $H_\infty$ is strictly decreasing, we have 
\begin{equation*}
 H_\infty(s_\infty-\ep)>H_\infty(s_\infty)>H_\infty(s_\infty+\ep). 
\end{equation*}
By the strong law of large numbers, $H_t(s_\infty\pm\ep)\to H_\infty(s_\infty\pm\ep)$ a.s. So, for some random variable $T_\ep$ such that $P(T_\ep\in\N)=1$ and all natural $t$, on the event $\{t\ge T_\ep\}$ we have 
\begin{equation*}
 H_t(s_\infty-\ep)>H_\infty(s_\infty)=0=H_t(s_t)=H_\infty(s_\infty)>H_t(s_\infty+\ep),
\end{equation*}
which yields
\begin{equation*}
 s_\infty-\ep<s_t<s_\infty+\ep,
\end{equation*}
because the function $H_t$ is strictly decreasing. Thus indeed, $s_t\to s_\infty$ a.s.
