Yes, take the the trivial connection with respect to the left trivialisation of the tangent bundle. Then, all of your curves $\gamma$ are geodesics, but there are no further geodesics.
Some more details: the tangent bundle of every (matrix) lie group is trivial: consider a basis $e_1,..,e_n$ of you Lie algebra, and the globally well-defined frame $X_1,..,X_n$ determined by $X_k(g)=g e_k$, where the right hand side is just the matrix product (if you have a matrix Lie group). Then you can define a unique connection by declaring the frame to be parallel. Equivalently, this means that all Cristoffel symbols vanish identically. This gives you a flat connection (with non-trivial torsion unless the Lie algebra is commutative, ie the lie bracket vanishes). Then, for any $v$ in the Lie group, the vector field $g\mapsto gv$ is parallel. Hence, your curves $\gamma$ are all geodesic.
If you want a torsion free connection, just add half of the commutator as a global connection 1-form. As this is skew, you get the same geodesics, but by construction this connection has vanishing torsion.