Try This Week's Finds in Mathematical Physics first. It has some classic references, e.g. Atyiah'sAtiyah's book:
The Geometry and Physics of Knots, by Michael Atiyah, Cambridge U. Press, 1990.
(update: AtyiahAtiyah has many books and to my knowledge any of them is worth a look)