# Mathematical physics applications in present-day image processing

During the past few years several important areas of image processing and image classification or generation became dominated by convolutional neural networks.

I'm interested if there are any methods coming from mathematical physics context (like methods designed for solving ill-posed problems, spectral analysis, image deblurring and deringing), that outperform neural network-based approaches in 2017 for some common computer vision or image processing problem. Or methods that don't have any neural network-based rivals. Maybe in the field of biomedical image analysis (just a guess)?

The deeper and more specific the answer is, the better.

## 2 Answers

Neural networks are best-in-breed at solving a very specific kind of problem: computing a function $f \colon X \to Y$ given the values of $f$ on a large but finite subset $A \subseteq X$. Typically $Y$ (the space of "labels") is finite and small relative to $A$. Concrete example: $X$ is the space of images, $Y$ is a set of labels for images (e.g. "contains a car" or "is a picture of a sunset"), and $A$ is a large set of human-annotated images (the training data).

As it turns out, neural networks can learn $f$ so well that they can produce points in $f^{-1}(y)$'s near a prescribed $x \in X$ - that's why they are good at image / language generation.

This is certainly an important and broad class of problems (particularly for applications in industry), and in practice neural networks don't really have any serious rivals. But there are lots of other image processing problems for which they aren't really appropriate - compression, recovery, noise reduction, etc. I don't have a lot of expertise to draw from, but for far as I am aware the standard techniques from physics and engineering are close to the state of the art.

I'll conclude by remarking that there are some persuasive arguments that the practical effectiveness of neural networks is explained by principles in physics. There is theory which says that if $X$ is large (e.g. all images or all sequences of characters in an alphabet) then no machine learning algorithm can perform better than the most naive ones. So to understand why neural networks are so high-performing one must use something about the structure of the underlying data sources (photographs or human language) to restrict $X$. Here are some recent papers in this direction:

https://arxiv.org/abs/1608.08225

https://arxiv.org/abs/1410.3831

• You talk about plain feedforward networks, but there are plenty of other architectures suitable even when you don't have any labels so maybe it's not as narrow as it seems. I'm almost sure that autoencoder networks are state-of-the-art in image compression, SRGAN ans SRCNN architectures are state-of-the-art in detail recovery/adding details. Not sure about denoiser networks but it also seems like an appropriate problem to tackle with neural networks nowadays (in absence of special limitations). – nikkou Apr 27 '17 at 19:15

Compressed sensing is an image processing technique, with many applications in particular in the context of MRI, that uses advanced math (based on contributions of one regular MO user), without a competing approach in neural networks.

• A standard reference that should be mentioned is Donoho's book under the same title. – Henry.L Apr 27 '17 at 17:52