For complete Riemannian manifolds the exponential map is a covering map at all points if
and only if the manifold has no conjugate points. In particular, the exponential map is a diffeomorphism at all points if and only if the manifold is simply-connected and the metric has no conjugate points.

Manifolds without conjugate points have been studied by many authors and they need not have everywhere nonpositive sectional curvature.

On the other hand it is still an open problem (as far as I know) whether every closed manifold without conjugate points admits a metric of nonpositive sectional curvature.

For one result relevant to your question ~~is that if a complete Riemannian manifol has no conjugate points, then either it is flat (that is has zero sectional curvature everywhere) or it has negative Ricci curvature everywhere;~~ see Geodesics without Conjugate Points and Curvatures at Infinity
by SÉRGIO MENDONÇA and DETANG ZHOU. They also mention some related result of Leon Green involving scalar curvature.

EDIT: I misread the curvature condition in the paper linked above. Thanks to Sergei for correcting me!