When I was first learning QFT, I found it very helpful to start in the parts that rest on the most solid mathematical foundations, like topological quantum field theories and 2d conformal field theories. This might be an especially good place to start if you're aiming for the geometric applications of QFT, because I think many of those applications currently revolve around TQFTs and CFTs.
There are lots of places you can learn about TQFTs.
- My favorite introduction is Joachim Kock's Frobenius algebras and 2D topological quantum field theories, which starts with the definition of a TQFT and builds up to a detailed proof of the first big theorem in the subject: the classification of 2d TQFTs.
- For a quicker introduction, you might try these seminar talks by John Baez, Julie Bergner, Chris Carlson, and John Huerta. PDF notes are linked from these pages: 1 | 2 | 3 | 4 | 5.
- For a taste of how TQFTs can be used to do geometry, try using using a topological gauge theory with finite gauge group $G$ to count principal $G$-bundles over a compact surface. Ulrich Pennig has some lovely notes on this. There's also a relevant MO question.
- For a taste of how TQFTs can be used to do representation theory, I recommend Qiaochu Yuan's excellent notes on a field-theoretic proof of Mednykh's formula.
- When you're ready to leave the 1-categorical nest and start thinking about extended TQFTs, try David Ben-Zvi's Northwestern lectures, as recorded by Orit Davidovich and and Alex Hoffnung. PDF notes are linked from these pages: 1 | 2 | 3.
- For a really serious application, you can try Turaev's Quantum Invariants of Knots and 3-Manifolds, which discusses the TQFT that Witten discovered for the Jones polynomial. It's tough reading, though.
- I love John Baez's "Quantum Quandaries" for the physics inspiration it provides, though maybe that's not your thing.
Here are a handful of references for CFTs that I like.
The classic one is Graeme Segal's The Definition of Conformal Field Theory.
Yi-Zhi Huang's Two-Dimensional Conformal Geometry and Vertex Operator Algebras is probably very relevant.
Liang Kong's "Conformal field theory and a new geometry" looks like a fun source of physics inspiration, though I haven't read it.
Finally, I should make it clear that there's lots of cool geometry related to QFTs more complicated and mysterious than TQFTs and CFTs.
- For a glimpse, you can try Yuji Tachikawa's "A pseudo-mathematical pseudo-review on 4d $\mathcal{N} = 2$ supersymmetric quantum field theories." It's tough going, though.
There's also lots of cool math related to QFTs more like the one's you'd encounter in an introductory physics course, although much of it is more analytic than geometric. Some references:
- N. M. J. Woodhouse's Geometric Quantization.
- Brian Hall's "Holomorphic Methods in Quantum Physics," which is not about QFT per se, but is good preparation for the infinite-dimensional theory described in...
- Irving Segal's "Mathematical characterization of the physical vacuum for a linear Bose-Einstein field." Section 7 is where this paper connects with Hall's review.
- For a much deeper treatment, see John Baez, Irving Segal, and Zhengfang Zhou's Introduction to Algebraic and Constructive Quantum Field Theory.
Good luck!