You will have to choose:
Do i want to program arrays, pixels, and compile standalone apps? do i want to use limited xy graphics and start nice and simple?
A language is more descriptive than maths, maths is a script, so it doesnt stream data into graphic forms, it doesnt have time and debug. You will have to make concessions to a framing language, to describe how you want maths to be streamed, memorized, displayed, debugged.
I'd say start obsessively and simply with a language that draws you in by it's fast results. You may find your brain suddenly expands in a strange way due to the breadth and depth of a new power to program anything you want, when you have written/understood your first 50 lines, which is all it takes to decide to continue...
You can specialize in anything your career draws you to, so CUDA/glsl if you want to do parallel processing and supercomputing, its very vogue, and specialized.
Shadertoy.com has 3d stuff, and vironoi, noise, graphics.
Matlab for pure maths, personally i found it cumbersome and dry.
c# for DSP, FFT, GameOLife, Algorythms, in unity3d is a fun environment. Paul bourke is a renowned mathmatician, dont know his primary code. C#?
Stuff like wigner quantum audio time frequency distribution or fractals are mostly in C family for legacy journal eork, thats why i like c#.
Some people do in depth topologies on openscad, like kitwallaces pages...
And k3dsurf has isosurface descriptions which are a bit futile but amazingly clever. Its pure trigonometry volumes only language.
K3dsurf and openscad are starter languages which can be learnt in 2 hours/days.
I learnt on reaktor visual math environmeht, then milkdrop, then javascript and scad.
Java isnt great these days, memory concerns, it used to be a top choice.