It may be helpful to consider an example that can be solved exactly; Using the <A HREF="https://arxiv.org/abs/1501.02506">Special-case closed form of the Baker-Campbell-Hausdorff formula</A> one finds that if the commutator $[X,Y]$ evaluates to
$$[X,Y]=uX +vY +cI$$
then the desired logarithm of the product of matrix exponentials equals
$$\log(e^X e^{tY})=tY+X+f(u,v,t)(uX +vY +cI)$$
$$f(u,v,t)=\frac{(ut-v)e^{ut+v}-ute^{ut}+ve^{v}}{uv(e^{ut}-e^{v})}$$
The large-$t$ limit can now be read off once the sign of $u$ is known:
$$\lim_{t\rightarrow\infty}f(u,v,t)=\begin{cases}
-1/u&\text{if}\;u<0\\
(t/v)(e^v-1)&\text{if}\;u>0\\
(t/v)[v-1+v(e^v-1)^{-1}]&\text{if}\;u=0
\end{cases}$$
There are no terms greater than order $t$ in the large-$t$ limit for this class of commutators. *(I'm actually a bit puzzled how $t^2$ terms and higher might appear at all, an explicit example would help me a lot.)*