Partly this depends on what you mean by "becoming a mathematician." I'd encourage you to think concretely about what sort of job you'd be happy with and figure out realistically what the odds are that you can get there from where you are.
For example, suppose "mathematician" to you means "tenured professor at a research 1 institution." Realistically this means you'd need to 1) Get a masters at a solid state school (you're unlikely to get into a top PhD program yet given your self-description), 2) Be the best student there within a few years, 3) Use that to get into a top 30 graduate program, 4) Depending on the caliber of the school be the best student, or at least the top handful, 5) Do very good research during your postdoc. That's a tough row to hoe, and although I don't know you personally, it's not something that a lot of people could pull off.
On the other hand, there are lots of other mathematical jobs out there. If you have a more broad set of jobs that you'd be happy with then it'll be easier for you to get there from where you are now. Certainly it'd be a reasonable goal to get a Math PhD somewhere, and end up with a job that involves mathematics.
Update: A good case study that illustrates both the possibility and the difficulty of becoming a very successful research mathematician after switching when not having top notch academic credentials, see William Stein's autobiography. Note the subtext of the discussion of his staying a year at NAU. He must have ended up with him getting basically a perfect score on the GRE and all of his professors must have said in their recommendations that he was by a significant margin the best student ever to have attended their school. A great thing about how few good graduate schools there are, is that even though a school like NAU's very rarely has students of Stein's caliber, nonetheless many of the profs went to good graduate schools (Wisconsin, Washington, Utah, etc.) and so understand what it takes to be successful at a good school. Thus they can credibly compare their once-in-a-blue-moon students to intro level students at a good graduate program.