I would love to understand the famous formula $g_{ij}(x) = \delta_{ij} + \frac{1}{3}R_{kijl}x^kx^l +O(||x||^3)$, which is valid in Riemannian normal coordinates and possibly more general situations.

I'm aware of 2 proofs: One using Jacobi fields [cf. e.g. S.Sternberg's "Curvature in Mathematics and Physics" from which the question title and formula is stolen :-) or cf. S.Lang's "Differential and Riemannian Manifolds"]. The other proof involves computing that $\partial_k\partial_lg_{ij}(x)$ shares some symmetries of curvature [cf. M.Spivak's "A Comprehensive Introduction to Differential Geometry, Vol. 2" where it is a several page "hairy computation" or cf. H.Weyl's 1923 edition of Riemann's Habilitationsvortrag (reprinted in a recent German book by Jürgen Jost) which I find uncomprehensible.]

**Are you aware of any other proof? Are normal coordinates necessary?**

While the Jacobi fields proof is short and elegant enough, it irks me that it requires "higher technology" not involved in the endproduct. Somehow the formula should be provable by pure calculus. Indeed, it is stated as an exercise in P.Petersen's "Riemannian Geometry": From the context I guess he thinks it should follow from the expression of $\partial_lg_{ij}$ as a sum of 2 Christoffel symbols and the simplified expression for curvature at $x=0$ where the Christoffel symbols vanish. Alas my attempts at this go in circles...

I find the situation quite amazing: Not many textbooks treat this fundamental and historic formula. (Estimating from the sample on my shelf it is 3/17. E.g. it seems it's not even in Levi-Civita's classic.)

**Update/Scholium:**

In classical language: The knackpoint seems to be a "differential Bianchi formula" for the Christoffel symbols at $0$. This follows from the geodesic equation. I see no other way yet.

A more modern approach minimizing (but not eliminating) the role of geodesics is in A.Gray's Tubes book. (Noted in comments. I'm waiting for www.amazon.de to deliver this treasure.)

$\bullet$ While geodesics are very geometric and normal coordinates are very practical, methinks the formula is a tad ungeometric. *What I'm hoping/asking for is a coordinate-independent formula for the second derivative of $g$ in terms of a suitable "reference connection".*

Tubescontains in Section 9.1 (2nd Edition) a readable presentation of the Taylor expansion of a tensor in geodesic coordinates. It includes the case of the metric tensor as a special case (Corollary 9.8). – Liviu Nicolaescu Oct 28 '14 at 14:34