I have two favourite books on differential geometry where you can find answers to your questions:

1. do Carmo's Riemannian Geometry (as suggested by David Lehavi)
2. Besse's Einstein manifolds

Let me just point out that your 4th point is not quite correct. The statement is that

A riemannian manifold is locally symmetric if and only if the Riemann curvature tensor is parallel with respect to the Levi-Civita connection; i.e., $\nabla R = 0$

This presupposes that by "locally symmetric" you understand that the geodesic symmetry (i.e., changing the sign of the parameter of the geodesic) is an isometry at every point.

I have two favourite books on differential geometry where you can find answers to your questions:

1. do Carmo's Riemannian Geometry (as suggested by David Lehavi)
2. Besse's Einstein manifolds

Let me just point out that your 4th point is not quite correct. The statement is that

A riemannian manifold is locally symmetric if and only if the Riemann curvature tensor (and not just the scalar curvature) is parallel with respect to the Levi-Civita connection; i.e., $\nabla R = 0$

This presupposes that by "locally symmetric" you understand that the geodesic symmetry (i.e., changing the sign of the parameter of the geodesic) is an isometry at every point, otherwise it is a definition of locally symmetric.

Edit (in response to Anirbit's comment)

This is indeed a result of Élie Cartan and in fact, as far as I understand the history, Cartan started his research on symmetric spaces by studying the question of which riemannian manifolds have parallel curvature. He then classified the irreducibles and found the well-known relationship to the classification of simple Lie algebras. I'm not sure when the characterisation in terms of the geodesic symmetry was introduced. The proof is not complicated. It is basically that the curvature tensor is invariant under the map which interchanges opposite points along a geodesic. In other words, if you fix a point $p $ in your manifold and look at a geodesic $\gamma$ through $p$ in the direction $X$, then if you follow the geodesic a 'time' $s$ you get to some point $p(s)$. But there is also a geodesic through the same point with direction $-X$ and if you follow that geodesic for a time $s$ you end up at a point $p(-s)$. The map which sends $p(s)$ to $p(-s)$ for any (small, say) $s$ leaves the curvature invariant. The covariant derivative of the curvature along $X$ at $p$ can be understood as the difference between the curvature parallel transported to $p(s)$ and that transported to $p(-s)$ divided by $s$ in the limit as $s\to 0$, but even before you take the limit, the difference vanishes.

Since you said your background is in relativity, I wonder whether you are not also interested in the case of locally symmetric spaces in lorentzian (or other indefinite) signature. In general signature this is still an open problem, but for lorentzian it was solved by Cahen and Wallach in this paper.