What's about "quantum modular forms"?  Do you know where one could read on "Modular Forms, K-theory and Knots"? The combination of themes sounds thrilling! 
Edit: Zagier's paper on "quantum modular forms" will be published in Clay's volume dedicated to Connes anniversary.  
Edit: Copies of the fascinating article circulate in the web. If asked by email, I would help finding it. 
Edit: a survey + lecture notes by Ken Ono on harmonic Maass forms. 
 A: There is an article Conformal Field Theory and Torsion Elements of the Bloch Group by W. Nahm in the book Frontiers in Number Theory, Physics, and Geometry II. This is a reference for one direction in the abstract of the talk by Zagier.
A: I'm not sure how K-theory enters in this context, but modular forms with half-integral weight are closely related to the Witten-Reshetikhin-Turaev (WRT) invariant of 3-manifolds. The story starts with a 1999 paper by Ruth Lawrence and Don Zagier. They calculated a radial perturbative expansion of the WRT invariant of the Poincaré homology sphere $\Sigma(2,3,5)$ with gauge group $U_\xi \mathrm{sl}_2$ at a root of unity, identifying it with the Eichler integral of the modular form with weight $\frac{3}{2}$. This is a tremendous observation, because it means that we can obtain the exact perturbative expansion by the modular property- which in itself is a dream come true- and then reinterpret parts of this expansion (Reidemeister torsion, Casson invariant...) in terms of the modular form.
This was the beginning of a long and fruitful topic of research, whose later heros included Hikami, J. Murakami, Habiro, and many others. See on MathSciNet.
Another reference which might interest you (again by Don Zagier) is 
D D. Zagier, Vassiliev invariants and a strange identity related to the Dedekind eta-function, Topology 40(5) (2001), 945-960.
This concerns a different relationship between quantum invariants and modular forms (in this case the Dedekind eta-function). Stoimenow defined a space of "regular linearized chord diagrams", whose rank in each degree could be explicitly computed to give an upper bound (the best currently known) for the number of linearly independent Vassiliev invariants of fixed degree. These same numbers are obtained also as the “derivative of order one-half ” of the Dedekind eta-function, for reasons described in the paper. I don't know what larger role "regular linearized chord diagrams" play in quantum topology, therefore I don't know how this observation is related to the first, nor how it fits into a wider picture.
The bottom line is that the relationship between modular forms and quantum invariants of 3-manifolds has been an active and fruitful topic of research for the last decade, and is likely to continue to be so in the next.
