$\newcommand\R{\mathbb R}$
It$\newcommand\R{\mathbb R}$It does not make sense to define a convex function on the nonconvex set $[-\infty,\infty]^m$.
Even if you replace $[-\infty,\infty]^m$ by $\mathbb R^m$ and even you just take $m=1$, then your desired statement will still be false in general. For instance, do take $m=1$ and let $f(x):=e^x$ and $f_n(x):=e^x\,1(x\ge-n)+e^{-n}(x+n+1)\,1(x<-n)$ for real $x$ and natural $n$. Then $f$ and the $f_n$'s are convex, $f_n\to f$ pointwise on $\R$ (as $n\to\infty$), but $$\inf_\R f_n=-\infty\not\to0=\inf_\R f.$$
A necessary and sufficient condition for the infimum-stability for convex functions $f$ defined on $\R$ was given by Theorem 1 of Theorem 1A Necessary and Sufficient Condition on the Stability of the Infimum of Convex Functions (one can also use the arXiv version).
The proof of the just mentioned result takes about 4 pages. No extension of that result to $m>1$ seems to be known.