Some of the recently emerged fields in machine learning have a bit of overlap with mathematics (not sure how pure they are). I'm going to name a few that comes to mind:

• Graph theory: a recently introduced network architecture known as graph neural network can be considered a generalization of belief propagation networks on structured graph. This may in addition have some overlap with statistical physics.
• Algebra/geometry: equivariant neural networks require some sort of explicit/implicit symmetry built into each layer, and very recently people have studied this type of network with symmetry related to certain Lie algebras
• Topology/measure theory: to give a marginally related example, normalization flows are neural networks that attempts to "continuously" deform a Gaussian measure to some other non-trivial (usually multi-modal) measures. For instance, people have used the Banach fixed point theorem to show such networks are actually "trainable".
• Complexity theory: transformers are a type of network that requires quadratic memory and compute to perform inference. Recently, people have investigated ways to reduce its complexity theory methods such as hashing and kernel methods.
• Optimization theory: currently the way neural networks are trained are somewhat ad hoc, and people just use whatever optimizers (e.g. Adam, SAM) that gives the best empirical results. Recently, people have started looking into this more seriously, and neural ODE is a type of network that can be trained via the Pontryagin method.
• Random matrix theory: the neural network layer weights can be considered as a random matrix, and the heavy-tailness of such matrices have recently be studied as indicators for the "complexity" of the network (and whether it is prone to overfitting).
• Dynamical systems: a group in UWashington are looking at ways to interface machine learning with dynamical systems. For instance, one direction is to use neural networks to discover implicit low-rank structures of nonlinear dynamical systems, such as SINDY for the Navier Stokes.
• Fourier analysis: there is a line of research that tries to convert convolution networks into recurrent networks, by apply Fourier transforms (or polynomial decomposition) to the network inputs and kernels. Many theoretical problems are still open, such as the stability and convergence of such conversion.

However, similar to the case of physics lagging behind mathematics for 50 years or so, most ML fields further lags behind physics 20-30 years. So I wouldn't count on using a ML-related job (research or industry focused) as a medium to gain immediate access to novel mathematical research. Rather, I'd view it as an opportunity to apply your own mathematical knowledge (instead of advancing it).