I studied mathematics at the undergraduate level, but I was always interested in computers and I spent my days as an apprentice at a super-computing center. Then I studied a mix of computer science and mathematics for my PhD at a computer science school. Nowadays I am a sort of amphibian: I work in a math department, my colleagues think I am a computer scientist, and my interests span from programming languages to homotopy type theory.
I can recommend doing math as a major at the undergraduate level first, and then switching to computer science later. It's not that hard to do, and having good general background in mathematics is a huge advantage in theoretical computer science. You never know when someone is going to come up with elliptic curves in cryptography, or homotopy theory in type theory, or advanced linear algebra in data mining, or Fourier transforms in video encoding, etc. I never heard any computer scientist say that they knew too much math. The math you learn as an undergraduate will stay with you. It will be much more difficult to learn new math later in your life.