There are lots of subject specific packages written by mathematicians out there: GAP, PARI/GP, SnapPea, Maccauley Macaulay (1 and 2), Magma, Singular, etc. Sage is a new python based open-source project that is trying to absorb these, its rate of progress is better in number theory than in other subjects, but Sage is the best thing to learn if you're starting from scratch. It's also Python based.
There are also commercial programs: Mathematica, Maple, Matlab, etc. In the circles I run in Mathematica is the most popular.
Of course for things where you need serious speed people tend to use C++, Java or some other lower level language.