Dear Toussaint, Frankel has written a book whose very title The geometry of physics: an introduction makes me think it might be what you want. The second paragraph of the introduction confirms this impression:This is a textbook that develops some of the geometrical concepts and tools that are helpful in understanding classical and modem physics and engineering. (the last word will warm your heart!)
Tensors and exterior products, for example, are presented in a very concrete way, without introducing more abstract concepts like quotients of free modules. And the pictures are numerous and very evocative.
Google will let you browse [the book] 1
To mathematicians Like many of us, I have a melanchcolymelancholy love for physics from my student days and I find this book written in exactly the right language for a mathematician wanting to (re)learn some physics (the title doesn't say it but there is quite a bit of physics in this book)