I recently read a short story called "Space" by John Buchan (author of The Thirty-Nine Steps) which is about a mathematician who discovers a gateway to the fourth dimension or something. It's quite good.
Also, Charles Kingsley wrote a novel about Hypatia of Alexandria back in the mid-nineteenth century. I haven't read it yet, but I found two copies in a secondhand bookshop the other day. I would be surprised if it didn't have some mathematics in it.
Edit: I just finished reading "Hypatia". Surprisingly, it didn't have any mathematics in it. It did contain this nice quote, however:
In the hour of that unrighteous victory, the Church of Alexandria received a deadly wound. It had admitted and sanctioned those habits of doing evil that good may come, of pious intrigue, and at last of open persecution, which are certain to creep in wheresoever men attempt to set up a merely religious empire, independent of human relationships and civil laws; to 'establish,' in short, a 'theocracy,' and by that very act confess their secret disbelief that God is ruling already.