My first question on Math Overflow.
For my Mathematics Bachelor thesis I am looking at a paper called "Deep Limits of Residual Neural networks" by Matthew Thorpe and Yves van Gennip. (arxiv.org link)
In Lemma 4.6, a preliminary result to show that something Gamma-converges, they first show that a discretized ODE uniformly converges to the ODE itself.
Some info you need:
- $X: [0,1] \to \mathbb{R}^d,$
- $\sigma$ is Lipschitz continuous with constant $L_{\sigma}$,
- $\dot{X}(t) = \sigma(K(t)X(t)+b(t)), \quad t\in[0,1], K: [0,1]\to \mathbb{R}^{d\times d}, b: [0,1] \to \mathbb{R}^d$.
The inequality that I cannot confirm:
$$ \sup_{s\in[0,t]} || \int_0^s \dot{X}(r)dr + x|| \leq \sup_{s\in[0,t]} L_\sigma s ||X||_{L^{\infty}([0,s])}||K||_{L^{\infty}} + L_{\sigma}||b||_{L^{\infty}} + ||x||. $$
Now I expected $||b||_{L^{\infty}}$ to also be scalar multiplied by $s$. Here are my steps
\begin{align*} \sup_{s\in[0,t]} || \int_0^s \dot{X}(r)dr + x|| & \leq \sup_{s\in[0,t]} || \int_0^s \sigma(K(r)X(r)+b(r)) dr|| + ||x|| \\ & \leq \sup_{s\in[0,t]} L_{\sigma}|| \int_0^s K(r)X(r)dr + \int_0^s b(r) dr|| + ||x||\\ & \leq \sup_{s\in[0,t]} L_{\sigma}\Big( \int_0^s ||K(r)X(r)||dr + \int_0^s ||b(r)|| dr\big) + ||x||\\ & \leq \sup_{s\in[0,t]} L_{\sigma}\Big( ||KX||_{L^{\infty}} \int_0^s dr + ||b||_{L^{\infty}}\int_0^s dr\Big) + ||x||\\ & \leq \sup_{s\in[0,t]} L_{\sigma}\Big( ||K||_{L^\infty}||X||_{L^{\infty}}\cdot s + ||b||_{L^{\infty}}\cdot s\Big) + ||x||. \end{align*}
So I think the $s$ in front of $||K|| ||X||$ comes from integrating over $\mathcal{1}$, and therefore the same thing should happen for $b$.
Also: In the inequality in the paper they use the $L^{\infty}([0,s])$ norm for $X$ and the $L^{\infty}$ norm for $K$ and $b$ on the right side, but I cannot explain this. Could this have something to do with the missing $s$?
