There are some speculative mathematical concepts that come to mind, such as the field of one element or motives, though perhaps these are more classifiable as "potential future mathematics" rather than "not mathematics at all", and certainly these speculative topics have at least *inspired* the creation of mainstream, commonly-accepted mathematics (rigorous theorems, applications to other fields of mathematics, precise conjectures, conceptual reworkings of existing theories, etc.). [And motives may be currently in transition from "potential future mathematics" to "actual mathematics"; I'll leave it to experts in the area to weigh in further on this.]

A more controversial example might be inter-universal Teichmüller theory, where there is genuine debate as to whether this is "actual mathematics", "potential future mathematics", or "not mathematics at all".

If one turns from subfields of mathematics to modalities of mathematics, then in the recent past there were some debates as to whether experimental mathematics or computer-assisted proofs counted as "real" mathematics, but I believe that the prevailing consensus nowadays (by which I mean in the last decade or so) is that these do broadly fall inside the realm of mathematics. (Though perhaps these debates may be re-ignited in coming years if AI-generated conjectures and/or AI-generated proofs of new mathematical theorems become commonplace.) Going back even further in time, we of course have some venerable debates about the use of non-constructive methods (cf. Gordan's quote on Hilbert's proof of his basis theorem being theology rather than mathematics), set-theoretic infinities, non-Euclidean geometry, complex numbers, etc., though again the modern consensus is very strongly in favor of classifying all of these methods and concepts as being part of the field of mathematics. (cf. Gordan's later quote - reported by Klein - on having convinced himself that theology has its advantages.)

Finally, in the 1990s, the topic of Bible codes / Torah codes did briefly attract some academic mathematical interest (and controversy), but it would be a stretch to consider it a "field of academic mathematics" currently.

EDIT: in the converse direction, there are certainly disciplines that are typically housed outside of academic mathematics departments that have a strong case of being considered to be primarily mathematical in nature. Theoretical computer science is one example that comes to mind; there may well be others.

SECOND EDIT:
Section 19 (Mathematical Education and Popularization of Mathematics) and Section 20 (History of Mathematics) of the (2022) International Congress of Mathematicians are both devoted to fields which one could certainly argue do not have the epistemic status of mathematics, but are still perfectly valid fields of academic study, and which are the primary or secondary interests of a non-trivial number of faculty at mathematics departments. Whether they qualify as "fields of academic mathematics" depends on one's definitions, though.

THIRD EDIT: The Online Encyclopedia of Integer Sequences (OEIS) is not, strictly speaking, a field, but it does have an active community of both professional and amateur mathematicians contributing to it, and is widely used within the academic mathematical community. One could pose the philosophical question of whether contributing to the OEIS is an activity that can be ascribed the epistemic status of "mathematics". Similar questions could be asked for the communities centered around developing mathematical software, such as proof assistants. However, my personal view is to incline towards a "big tent" view of mathematics, and that excessive gatekeeping of what qualifies as "genuine" mathematics could be harmful towards achieving progress in the field.

"some people think [this] is a mathematical question with a mathematical answer"? $\endgroup$professional string theoristswho believe that string theory is not physics. $\endgroup$17more comments