There is a "classic" book about the history of mathematical notations by Florian Cajori though there has been some "revision" of his work by more recent scholars.

http://en.wikipedia.org/wiki/Florian_Cajori