It is well-known that many great mathematicians were prodigies.
Were there any great mathematicians who started off later in life?
Joan Birman went back to grad school in math in her forties, and is now one of the top researchers in knot theory.
Karl Theodor Wilhelm Weierstrass (Weierstraß) Follow this link
According to this Notices article, Raoul Bott was undistinguished in high school, but displayed impressive talent once he reached graduate school (though his thesis was actually in electrical engineering, rather than mathematics).
I don't think that Stephen Smale really distinguished himself until after graduate school.
Eugene Ehrhart (of Ehrhart polynomial fame) was born in 1906, taught in various French lycees (high schools), began his work on geometry in the 1950's, did his best work in the 1960's, and received a Ph.D. in 1966. See http://icps.u-strasbg.fr/~clauss/Ehrhart.html.
Here, Rob Kirby describes some of his experiences as an undergraduate at Chicago, and how he "snuck into graduate school".
As an undergraduate, I'd been far more interested in chess, poker, and almost any sport, than in the game of mathematics. I had little chance of getting into a good graduate school. However, I failed German and didn't get a B.S. in four years, so in my fifth year I took most of the graduate courses on which the Masters Exam (really a Ph.D. prelim) was based. With a B.S. I asked to be admitted to graduate school so as to take the Exam. They cautiously said yes if I got grades closer to B than C in the fall quarter. I got a B and a C (measure theory from Halmos and algebraic topology from Dyer) and a Pass, and no one told me to leave.
The Masters Exam could have four outcomes: you could pass with financial aid, pass without aid but with encouragement, pass with advice to pursue studies elsewhere, and fail. I got the third pass, but really liked Chicago and turned up the next year (1960) anyway.
Sophus Lie didn't become interested in mathematics until after university, and before then didn't seem to have shown significant aptitude for it.
Somebody who probably fits the bill here is Albrecht Fröhlich who after fleeing Nazi Germany as a teenager, eventually attended university only when he was about 30. He later went on to jointly organize the Brighton conference which put class field theory on the mathematical map, essentially create a new branch of number theory and produce his most important work well into his fifties.
There's a biographical memoir here.
Preda Mihailescu is a good example http://en.wikipedia.org/wiki/Preda_Mihăilescu.
He received his PhD with 42, and proved the Catalan conjecture 5 years later.
The Catalan conjecture was open for 160 years.
He proposed in 2009 a proof of the Leopoldt conjecture, but I am not sure about the status of this.
Persi Diaconis of Stanford had two careers -- the first as a violin prodigy studying at Julliard, and then as a world famous magician who performed for the crowned heads of Europe. In his early twenties he decided that he wanted to learn enough math to understand Feller's two volume treatise, so enrolled at CCNY. His beginning was rocky (by his own admission), but he finished there well enough to be admitted to the Ph.D. program in Statistics at Harvard, and, the rest, shall we say, is history. He also worked as an advertising copywriter while he was attending CCNY.
I've just been reading Peter Roquette's entertaining account of the remarkable career of Otto Grün. Grün was an amateur, never attended university but at the age of 44 sent some results around FLT to Helmut Hasse. There were considerable errors, but Hasse spotted enough originality to keep up a correspondence and helped guide Grün into becoming a highly respected group theorist with work fundamental enough to find it's way straight into group theory text books.
How about Raymond Smullyan? According to his autobiography, he has published his first mathematical article at the age of 35, to which Marvin Minsky has reacted by saying Ray has decided to become a child prodigy at the age of 35. Does this count as starting off late in life?
 Raymond Smullyan, Emlékek, történetek, paradoxonok. TyopTeX, 2004, original title "Some Interesting Memories. A Paradoxical Life".
You could read the autobiography of Paul Halmos (RIP- he died just a few years ago) "I want to be a mathematician". He started mathematics much later in life, first he did chemical engineering then philosophy then mathematics. Halmos wasn't quite the genius in mathematics (as he has described it) but later in life he got into it and succeeded. John von Neumann (who was Halmos' countryman) also started from chemical engineering before going to mathematics.
One could perhaps also cite George Green, miller and mainly autodidact mathematician as an unconventional and relatively late bloomer. He entered University only at 40, one year or so before his death. See for instance http://www-history.mcs.st-andrews.ac.uk/Biographies/Green.html>Green's Biography at MathTutor
Alberto Calderón (of Calderón-Zygmund Theory/operators - one of the great analysts of the 20th century by any account). He studied Electrical Engineering in Buenos Aires, graduating at age 27. Zygmund met him during a visit to Buenos Aires and was very impressed with his mathematical originality, so he invited him to pursue a PhD in Chicago, which Calderón completed when he was 30.
According to an interview of Arnold in Notices (p437), both Whitney and Kolmogorov switched subject at university after a couple years and chose mathematics (Whitney was studying violin, Kolmogorov was into history). So they discovered math after high school, but the interview makes it clear both were very smart (not late bloomers).
There's Thomas Kirkman of Kirkman's schoolgirl problem, who didn't start studying mathematics until he was into his forties. Aside from the problem that bears his name he went on to work publish papers in extremal set theory, finite geometries and the like, he was also one of the first to write about group theory in English.
There's Serge Lang. Apparently, he finished his undergraduate degree in physics at CalTech, before a short tour of duty in Europe. When he returned for graduate studies, he was initially enrolled in Princeton's philosophy department. According to the biography, he switched to mathematics after his first year, and worked with Emil Artin.
A similar question was discussed some time ago over at http://quomodocumque.wordpress.com/2009/01/21/claimed-proof-of-the-abc-conjecture/#comment-3179. There it was pointed out that William H. Young (of, say, Hausdorff-Young fame) didn't publish much before 40. After 40 he had a successful career, with a prolific publication record. This said, it is reported that he was a very talented student.
In May 2006, the AMS Notices printed a remembrance article for Serge Lang. Dorian Goldfield was one of the contributors, and as an undergraduate, he described himself as follows:
Of the many people who had serious interactions with Serge, I am one of those who came away with fierce admiration and loyalty. In the mid-1960s, I was an undergraduate in the Columbia engineering school on academic probation with a C–average. In my senior year I had an idea for a theorem which combined ergodic theory and number theory in a new way, and I approached Serge and showed him what I was doing. Although I was only a C–level student in his undergraduate analysis class he took an immediate interest in my work and asked Lorch if he thought there was anything in it. When Lorch came back with a positive response, Lang immediately invited me to join the graduate program at Columbia the next year, September 1967.
Then again, Goldfield was not a "late learner" as he was 20 when he finished college and 22 when he earned his PhD. But...
Here's a great example:Makus Fisz. "Who?!?" An expert in probability and statistics who was born in 1910 and grew up in war-torn Poland-and as a result,his career kept getting interrupted. He finally got his doctorate at the age of 40 and published a number of well known papers as well as an acclaimed text on the subject that was translated into a half a dozen languages and became very popular in Europe.He was finally appointed full professor of mathematics at New York University after many visiting positions. Tragically,he died of a heart attack at the age of 54. A great story and career with a very sad ending.