Jeffrey Rabin has written a lightning-fast introduction to physics designed for exactly the audience you describe: people with "the mathematical background of a first-year graduate student," but "[no] prior knowledge of physics beyond F = ma."
It's a bit single-minded, because Rabin's ultimate goal is quantum field theory, but it hits most of the important subjects in modern physics, including:
- Classical mechanics (Newtonian, Lagrangian, and Hamiltonian formalisms)
- Classical field theory
- The Lorentz group (presumably that means some special relativity?)
- Quantum mechanics (not sure how in-depth this section is, but it's better than nothing)
The only glaring omissions I can see are classical electromagnetism, statistical mechanics, and general relativity.
You can find Rabin's introduction in the book Geometry and Quantum Field Theory.