As no one has given any references that aim to improve physical intuition, I would recommend Einstein's classic "The Meaning of Relativity". This short book gives good physical insights into the theories of special and general relativity, and the mathematics should be easy enough for someone with a knowledge of differential geometry. It contains a good portion of text that aims to explain Einsteins reasoning as the basis for the theories, as well as some more philosophical ponderings.
For someone less knowledgable in differential geometry, I think that Einstein's book, coupled with J.G. Simmonds' "A Brief on Tensor Analysis" would be a good combination. Although it is a math book, Simmonds goes a long way in offering physical intuition as a means of motivating differential geometry and tensor analysis for physics and engineering students. Also, this book is short, and at an undergraduate level, so the first two chapters should read extremely fast for a seasoned mathematician.
