I have a function $f:\mathbb{R}^n\to\mathbb{R}^m$. My goal is to bound the first order Taylor approximation of $f$. Given $x,x'\in\mathbb{R}^n$ I have that \begin{equation} f(x)-f(x')\approx (x-x')^TDf(x')+(x-x')^THf(x')(x-x')+\mathcal{O}\left(\left\Vert x-x' \right\Vert^3\right), \end{equation} where $Df(x')$ and $Hf(x')$ define the Jacobian and Hessian w.r.t $x'$, respectively. Taking the 2-norm of each side we get
\begin{align} \left\Vert f(x)-f(x')-(x-x')^TDf(x')\right\Vert_2&\leq \left\Vert(x-x')^THf(x')(x-x')\right\Vert_2\\ &\leq\left\Vert H f(x')\right \Vert_2\left\Vert x-x'\right \Vert^2_2 \quad (\text{operator norm}).\\ &\leq C \left\Vert x-x'\right \Vert^2_2\\ \end{align}
My goal is to bound (and find $C$) such that $\left\Vert Hf(x') \right \Vert_2\leq C$. The confusion I have is that as $f:\mathbb{R}^n\to\mathbb{R}^m$, the Jacobian and Hessian are both tensors. Is the correct approach to split $f=(f_1,f_2,\dots,f_n)$ and take $n$ separate Taylor approximations of $f_j:\mathbb{R}\to \mathbb{R}^m$ for $j=1:n$ and then we would have $\left\Vert Hf(x') \right \Vert_2^2=\sum_{j=1}^n\left\Vert Hf_j(x') \right\Vert_2^2$ where $Hf_j(x')$ is the individual Hessian of $f_j$ at the point $x'$ ?
Or am I missing something blindingly obvious here?
Thanks in advance.
Comments I posted it here but didn't get the answer I was looking for.
