As *HA Helfgott* suggests, at many schools, it is quite feasible to simply double-major in math and in computer science.

NYU, as you mention, offers a [joint major in math and computer science](http://math.nyu.edu/dynamic/undergrad/majors-minors/major-minor-requirements/#mathcsmajor), including courses in Partial Differential Equations,
Functions of a Complex Variable, and Abstract Algebra.

There are also programs in ["Mathematical and Computational Science," e.g., @Stanford](https://mcs.stanford.edu/).
Georgia Tech, another school which you mention, offers [a "thread" in Theory](http://www.cc.gatech.edu/theorythread), including courses in Number Theory and Cryptography,
Abstract Algebra, Linear Programming, Monte Carlo Methods, Quantum Information and Quantum Computation.

I think you will have more options than you can imagine by just looking at the major requirements. There are many opportunities for "electives" to pursue your interests (as *oxeiman* suggests).