All algorithms are based on mathematics, so if you want the word "mathematical" in your subject to be meaningful you need to be more specific.

Borwein's list in another answer is very heavily numerical. Here's a more combinatorial list, drawn from the topics I would typically cover in an undergraduate introductory algorithms class:

• integer arithmetic (including the elementary algorithms for addition, multiplication, etc but also faster divide-and-conquer multiplication, the equivalence in complexity between multiplication and division, modular exponentiation, and gcds; maybe also matrix multiplication and RSA cryptography)
• sorting (both comparison-based and integer sorting), searching sorted lists, and median finding
• pattern matching in strings
• dynamic programming (longest common subsequence, knapsack and subset sum problems, etc)
• graph algorithms (breadth first search, depth first search, topological ordering, minimum spanning trees, and shortest paths)
• computational geometry (nearest neighbors and convex hulls)