The CAT(0) asumption is sufficient: it implies that any two points are connected by a unique geodesic segment. This is well-known and follows from the definitions.

However, as pointed out by HenrikRüping, the CAT(0) asumption is not necessary, you can for instance perturb the hyperbolic plane by putting a small lump of positive curvature near a point and the resulting metric will not be CAT(0) but will still be uniquely geodesic.

You might however get a positive result in this direction if you add some smoothness and a (fairly strong) topological asumption, namely, that your space is the universal cover of a torus. Then you can use the result by Burago and Ivanov here that any Riemannian metric without conjugate point on a torus is flat. If a metric is uniquely geodesic then it is very close to having no conjugate points, and by their result it has to be flat, hence CAT(0). So as far as I can see it's conceivable that a uniquely geodesic distance on the universal cover of an $n$-dimensional torus is flat, hence CAT(0) (?)