As far as quantum invariants, there is Witten's Quantum Field Theory and the Jones PolynomialQuantum Field Theory and the Jones Polynomial, which gives a 3-dimensional definition of the Jones polynomial (usually defined, as you say, using 2-dimensional diagrams) at a physical level of rigor. I understand that this paper was extremely influential, but I'm not familiar with the details. More recently, Witten's Fivebranes and KnotsFivebranes and Knots gives a 5-dimensional definition of Khovanov homology at a physical level of rigor.