Disclaimer: I don't have aphantasia, so this may be irrelevant to your question.
I graduated university with a BS in math and am starting grad school this fall. I am not an aphantasiac, but when I learned about aphantasia I immediately related. My thought process almost never defaults to mental imagery, although I can make wispy, vague mental images with much effort which are gone almost the second they arrive. My thinking is almost completely semantic - dialogue, facts, data, concepts, etc. with no accompanying mental sensation. I actually sense that this is a strength for me. Visual representation, to me, is a huge distraction, especially in mathematics. As the geometric topologist pointed out, the way things (whether that be actual points in space or concepts) are related to each other, factually, is usually the meat of what you're looking for, and having a mental process that cuts right to that without any middle-man step in between "feels" more efficient. But maybe that's just because it's all I've ever known. I suspect that it's the reason I'm able to mentally handle large amounts of data at a time - because in my head it very rarely comes with any cumbersome (visual or otherwise) attachments.
A more practical example of this is the way I remember driving directions. I don't remember what the road looked like or possess any spatial feel or awareness of what my surroundings were, but I remember a series of facts. Example: pass two stop signs, then turn right... Instead of simply approaching the road and visually remembering, ah, this is where I turn right. I guess in that sense, any time I drive somewhere is like the first time again. I'm giving myself instructions in my head as if I'm giving them to someone else who needs directions.
Ironically, one of my strongest areas in my undergrad was graph theory. I know the factual characteristics of any certain graph, but it comes to my mind unencumbered by the visual representation, and I quite like that. It's more streamlined and "clean" to me. My weakest area was vector calculus; the professor constantly told us to just "see" the dimensions of whatever problem we were working on in the mind's eye. Since the discovery that the strength of one's ability to visualize may exist on a spectrum, I understand why that seemed nearly impossible and so frustrating to me. So it seems that I am more inclined toward anything dealing with relationship (or fact and pattern, like number theory) as opposed to problems dealing with measurement and real-world dimension.