Exponential map in your definition is closely related to the smooth family of smooth curves smoothly depending on the position such that in every point in every direction there exists precisely one curve at this point in this direction. In literature, such a family is sometimes called path structure.

There are two differences between your structure and path structure: the first one is that you curves are parameterised, and the curves of path structure are not. But this an additional structure, so my answer below will has sense also for your setup.

The other difference is that for a path structure there exists precisely one, or, in the case of irreversible path structures, at most two curves from the family. You definition allows that (for three points A,B,C) the curves from A and B to C come to C with the same velocity vector which is not possible for geodesics. Or, even more wild, take 3 points A, B, C and consider the trajectory $\gamma_{ABC} $
of your exponential map which starts from A and then passes through first B and then C (assume such trajectory exists, i.e., choose B and C on a trajectory from A). Your definition does not require that the segment of $\gamma_{ABC}$ from $B$ to $C$ is a trajectory of your exponential map starting from $B$ and going to $C$, though this property may be essential for geodesics.

But, returning to path structures, most path structures do not come from an affine connection (known since at least Cartan for dim 2, essentially the same proof works for all dimensions, see https://arxiv.org/abs/1101.2069). In dimension 2, every reversible path structure is locally Finsler-metrisable https://arxiv.org/abs/1002.0243, the question whether every irreversible path structures is metrisable is still open.

In higher dimensions most path structures are, even microlocally, not Finsler-metrisable, see (the open access paper) https://link.springer.com/article/10.1007/s10714-022-03006-2 .