When I teach the exponential of matrices, I tell the students that the converging series is not a practical tool for calculation. It is way better to solve the differential equation. This turns out to be true here. Say $T$ is upper triangular and $M$ denotes $\exp T$. For the sake of simplicity, set $t_i=t_{ii}$Then one finds
$$m_{ii}=e^{t_{ii}},\qquad m_{i-1,i}=t_{i-1,i}\frac{e^{t_{i}}-e^{t_{i-1}}}{t_{i}-t_{i-1}}$$
and
$$m_{i-2,i}=\frac{t_{i-2,i}t_{i-1,i}}{t_{i}-t_{i-2}}\left(\frac{e^{t_{i}}-e^{t_{i-2}}}{t_{i}-t_{i-2}}-\frac{e^{t_{i-1}}-e^{t_{i-2}}}{t_{i-1}-t_{i-2}}\right)$$
and so on. All the entries $m_{ij}$ are obtained by successive finite differences. As usual, when an argument is repeted, the finite diffrence is a derivative.