How can Machine Learning help “see” in higher dimensions?

Question

The news that DeepMind had helped mathematicians in research (one in representation theory, and one in knot theory) certainly got many thinking, what other projects could AI help us with? See MO question What are possible applications of deep learning to research mathematics?

Geordie Williamson has been giving talks on his experience with the DeepMind team (his lectures in Machine Learning for the Working Mathematician are also great), and his answer would be whenever we can produce lots of data or examples which we wish to look for patterns. He gave graphs and high-dimensional manifolds as examples.

For high-dimensional manifolds, it would be great if AI could provide the kind of geometric intuition that we acquire “easily” for 2 and 3 dimensions (and struggle even for 4). But what would be a good task for AI to learn? Predicting the (co)homology of manifolds or CW-complexes?

In fact, in the latest advances in natural language processing, they found that the vectors that certain words are assigned to satisfy some curious relations, like “queen - woman + man = king”. Maybe AI can learn that cycles can be added too.

One task that I’m having in mind now: given the complement of a complex hypersurface in $\mathbb C^N$, decide if two cycles are homologous. That’s something we could just “see” in $\mathbb C^1$, but somehow it doesn’t feel like a “fundamental” question for geometric intuition, but what geometric intuition helps to answer (if there’s a difference).

So, my question is, what tasks would be “fundamental” to forming our geometric intuition? And for those who acquired some intuition in higher dimensions (regular polytopes? sphere-packing? singular locus?), what would be a good dataset and task for acquiring that intuition, whether or not it’s feasible for current machine learning techniques?

Answer 1

This is a comment rather than an answer.

Mathematicians have vastly differing experiences in the mathematical world. Consider the phrase "let $M$ be a 3-dimensional hyperbolic manifold". Some (like me) know the definition and might have a vague picture of what this means. Some don't know the definition and run away. Others have an extremely rich intuition, experience with many examples etc. This intuition is very difficult to communicate, which is one of the reasons it takes 3-5 years to get a PhD! (See this MO question for a fascinating discussion of the difference between what we write and what we think.)

Furthermore, mathematicians have very different mental models for "the same" object. I sometimes work with group schemes. For several years I asked experts in my field what they "see" or "imagine" when they picture a group scheme. What I suspected was true: some experts think of a group scheme as "basically a group with sauce", others as "basically a scheme with sauce", others as a "machine producing groups from rings" etc. These are very different conceptions of the same object! These different conceptions mean that certain mathematicians are more likely to pick up on aspects "obvious" from their point of view.

As pointed out by my colleague Oded Yacobi, this parallels the sense to which animals can have different Umwelten. That is, animals existing in the same environment can be sensitive to very different aspects of that environment. Birds appear to pick up on magnetic fields, certain shrimp are sensitive to bigger chunks of the visual spectrum than other animals (including humans).

Now ask a shrimp "what do you see?" Or ask a bird "how do you know where to go in summer?"

I think this is a very interesting thought experiment. It is also quite close to what you are asking. One thing that is certain is that we won't be able to guess the answer in advance!

One more comment: my experience with ML so far suggests that my intuition for what will be "easy" and "hard" for a neural net are unreliable. I suspect that there are many interesting fields of mushrooms out there, but finding them will involve quite some wandering.

Answer 2

At the interface of topological data analysis and AI (neural networks, machine learning) there is a variety of applications that help to visualize high-dimensional data. One application in this context is Estimating Betti Numbers Using Deep Learning (2019)

This paper proposes an efficient computational approach for estimating the topology of manifold data as it may occur in applications. For two- or three-dimensional point cloud data, the computation of Betti numbers using persistent homology tools can already be computationally very expensive. We propose an alternative approach that employs deep learning to estimate Betti numbers of manifolds approximated by point clouds. The approach could be generalised beyond estimating the numbers of holes, cavities and tunnels in low-dimensional manifolds to counting high-dimensional holes in high-dimensional data.

Answer 3

The method "UMAP" comes to mind and uses a lot of mathematics from category theory, topology to reduce dimension. I have tried it on "real world" datasets and on "number theoretic datasets" with custom distances on natural numbers and it works as one would expect it.

Here is a video I found and like on yt, created with UMAP about the first million integers:

https://www.youtube.com/watch?v=nCk8dyU7zUM

Maybe you can see something after reducing the dimension with umap...