I may have a similar answer with an alternative derivation. Given a group $G$ with Lie algebra $\mathfrak{g}$ and right-invariant inner product $\langle A,B\rangle_g=\langle Ag^{-1},Bg^{-1}\rangle_e$ where $A,B\in T_gG$, $Ag^{-1}=(R_{g^{-1}})_*A$, $Bg^{-1}=(R_{g^{-1}})_*B$ and $e$ being the neutral element. Right-translations of a group are generated by left-invariant vector fields which are therefore Killing vector fields. Let us assume that $M_I\in\mathfrak{g}$ for $I\in\{1,\cdots,\dim{G}\}$ is a basis of the Lie algebra. This means we have the left-invariant vector fields
\begin{align}
X_I=gM_I=L_gM_I\in T_gG\,.
\end{align}
It is a well-known fact that the inner product of (affinely parametrized) geodesics $u: [0,1]\to G: t\mapsto u(t)$ have conserved quantities
\begin{align}
C_I&=\langle X_I,\dot{u}(t)\rangle_{u(t)}\\
&=\langle uM_I,\dot{u}\rangle_u\\
&=\langle uM_Iu^{-1},\dot{u}u^{-1}\rangle_e\,.
\end{align}
For a Lie group with right-invariant metric, knowing the $\dim{G}$ conserved quantities $C_I$ and initial point of a geodesics $u(0)$ determine the geodesics uniquely.
Let us check what the condition for a 1-parameter subgroup $u(t)=e^{tA}$ with $A\in\mathfrak{g}$ is to be a geodesic, or equivalently to have the $C_I$ being conserved. Plugging $u(t)$ into above formula gives
\begin{align}
C_I=\langle e^{tA}M_Ie^{-tA},A\rangle_e\,.
\end{align}
We can take the derivative $d/dt$ on both sides and find
\begin{align}
0=\langle e^{tA}[A,M_I]e^{-tA},A\rangle_e\,.
\end{align}
This formula is equivalent to $\frac{d}{dt}\lVert e^{tM_I}Ae^{-tM_I}\rVert=\langle[M_I,A],A\rangle=0$ that Claudio wrote above and that Denis guessed. It's just an alternative derivation without making reference to the Killing form or its properties...