This question was asked a long time ago and got several helpful comments, but remains on the unanswered queue. Let me try to answer, as someone trained in pure math who has also done work in data science.

First, this career transition is becoming increasingly common, and no PhD student should think of it as a failure in any way. Every year there are around 1500 PhDs in math, and only around 800 tenure track jobs (stats is a separate situation). Additionally, jobs in data science are very attractive, often paying more than a professorship, providing lots of interesting problems to work on, and even having great benefits like more vacation days than average in the USA (e.g., I've met Google employees who get 4 or even 6 weeks off per year, based on how long they've been with the company), or the ability to work remotely. In Europe, it's even harder to get a professorship (for one thing, they have fewer jobs like small liberal arts colleges, regional universities, and community colleges), so there's more of a cultural norm of getting your PhD in math and then going to work in industry. The field of pure math does not seem to matter a ton. I know lots of abstract homotopy theorists who are now data scientists, not using their homotopy theory at all. Most advisors just want their students to be happy and successful, and would not be upset if a student realized they don't want to be a research mathematician but then landed on their feet as a data scientist.

Once you've decided you want to make that career transition, you should get on LinkedIn and start building a network of people who work in the kinds of jobs you want. You can find such people as alumni of your university (some might even be younger than you, having gone from undergrad into data science), or from folks in your research area (who you might know from conferences) who have left academia. If you're in a city, there are often meetups where you can meet young professionals in data science (you can find these on Facebook or Meetup.com or LinkedIn, among other places). This can also help you with your people skills, which will be important for interviews. You might also be able to take advantage of your university's career placement/counseling service, e.g., to touch up your CV for non-academic jobs, or to do practice interviews.

You will also want to devote as much time as possible to developing the skillset required to be a data scientist. Keep doing your job as a PhD student (your TA work and research, since you said you do want to finish your degree), but also carve out time to learn how to program, to learn the basics of statistics, etc. You can probably audit undergraduate courses if you want to, but as a math PhD student you can probably learn the material at a faster pace on your own. Still, professors might be willing to share their course materials with you to help your self-study. I'd advise you to talk to your advisor about your career plans and discuss the minimum requirements to complete your PhD (rather than the strongest possible thesis to get the best possible postdoc). Now is not the time to be a perfectionist in your thesis writing. Just get it done, and leave time for working on your transition out of academia while you still have a salary and benefits (like health insurance).

You need some baseline skills and knowledge beyond what is normally part of a math PhD program. You might enjoy the book Data Science for Mathematicians (full disclosure: I wrote a chapter of the book, but get no money from sales). The skills you need for success in data science are:

• Computational/algorithmic thinking, i.e., how to come up with an algorithm to solve a problem, and then implement that algorithm in code. I'd focus on Python first as it's easier to learn than C++ and sufficient for many data science jobs.
• Statistical thinking, like the content of a good stats course at the undergrad level, plus how to fit and assess models in R. Even better if you can read a book about Bayesian stats, time series analysis, and statistical learning (aka machine learning aka data mining).
• Clustering
• Operations Research
• Linear algebra, machine learning (like PCA, SVD, separating hyperplanes), dimensionality reduction

Once you've got the basics, you might consider a data science bootcamp to really put theory into practice. It might also be wise to carry out these algorithms on various interesting real-world data sets, and build up a portfolio of work that you could show prospective employers via a link to your GitHub.

Depending on what, specifically, you find most appealing about data science, it could also be good to learn about the following topics (covered in the book Data Systems, of which I am an author, among many other places):

• Tidy data, and the pandas package in Python
• Relational data, like SQL
• Hierarchical data, like html files
• Web scraping, APIs, and security/permissions issues

Don't expect to learn everything before you even start working as a data scientist. Part of how you can market yourself is as someone who can pick up new skills on the fly, and implement them at a high level. Expect to be learning for many years to come (hopefully that appeals to you). Another good way to market yourself is as someone who understands linear algebra at a very deep level. Most of machine learning and data science hinge on linear algebra, and folks working in those areas know the value of a mathematician who can think about linear algebra from a variety of viewpoints, and can use linear algebra techniques (which can be implemented as fast algorithms) to solve a variety of hard problems. Mathematical/analytical thinking can also be useful on the software engineering side (e.g., producing good test cases, producing code that is guaranteed to work) and on the stats side (thinking deeply about issues of bias and how that affects the analysis, understanding statistical techniques/tests/models deeply enough to be able to modify them when one of the assumptions is not met).

Lots of people have successfully made this transition, and if you get on LinkedIn you can find them and learn from their experiences. Believe in yourself, work hard, and think of (and market) yourself as a success instead of as a failure, and you can have confidence that you will succeed and get a great job. Good luck!