As HA Helfgott suggests, at many schools, it is quite feasible to simply double-major in math and in computer science.
NYU, as you mention, offers a joint major in math and computer science, including courses in Partial Differential Equations, Functions of a Complex Variable, and Abstract Algebra.
There are also programs in "Mathematical and Computational Science," e.g., @Stanford. Georgia Tech, another school which you mention, offers a "thread" in Theory, including courses in Number Theory and Cryptography, Abstract Algebra, Linear Programming, Monte Carlo Methods, Quantum Information and Quantum Computation.
I think you will have more options than you can imagine by just looking at the major requirements. There are many opportunities for "electives" to pursue your interests (as oxeiman suggests).