Paradoxical Mathematical Objects Pending for Construction The possible properties and applications of some mathematical objects have been described far before their rigorous mathematical definition. Some of them even had a seemingly paradoxical description which prevented them from existing at the time. 
An iconic example is the imaginary number, $i$, which was originally described as $\sqrt{-1}$. However, further advancements in the field and exploring new perspectives towards the subject eventually led to a rigorous non-contradictory mathematical definition of $i$. A definition that serves as a formal introduction of the (already existing) concept to the mathematical literature while satisfying all of its required properties. 
Similar attempts have been conducted along the lines of providing a solid formal basis for seemingly paradoxical concepts such as negative probability or sets with a negative number of elements (see also multisets):
Loeb, D., Sets with a negative number of elements, Adv. Math. 91, No. 1, 64-74 (1992). ZBL0767.05005.
Another contemporary example of such paradoxical mathematical objects which are pending for a rigorous definition is the so-called "field with one element". It naturally appears in various occasions and relates different concepts together. For instance, a group can be seen as a Hopf algebra over the "field with one element". See also this related MO question for further information.

Question. I am looking for more examples of such deep and useful mathematical concepts with contradictory descriptions which are not rigorously defined yet (or at least have no widely accepted formal definition). References to their possible applications are also welcome. 

Remark. I would like to emphasize the initial paradoxical description of the possible answers to this question. The important point is that the natural appearance of paradoxical objects often indicates the urgent need for a deep expansion of our mathematical worldview or an essential improvement of the foundations.        
 A: Grothendieck surmised the theory of motives 50(?) years ago, but people are still trying to figure out what it is.  I'm not sure if there are "paradoxes" involved though.
A: The Dirac delta function is a "function" which is zero almost everywhere but integrates to one. No function can possibly behave like that, but it nevertheless seemed like a useful notion. It was eventually made rigorous by the theory of distributions (or generalized functions).
It shows up paradoxically if you try to figure out what the identity element for the convolution operator. You can quickly determine that no such element exist, but if it were to it would have such and such properties. Said properties are exactly the defining properties of the Dirac Delta.
A: Fractional derivatives has been defined and found useful in some contexts.
A: Path integrals are a standard example.  Rigorous path integrals can be formulated in many cases but there is still no completely general approach to path integrals that satisfies both physicists and mathematicians.
A: This paper of Parisi makes a case for the following two objects: $r$-dimensional Euclidean space for $r$ not an integer, and the space of linear transformations on that space (= $r\times r$ matrices with real entries). Parisi argues that these are ideas which ought to have meaning, but do not yet.
A: The self-avoiding walk was famously analyzed by de Gennes using a generalization of the Ising model known as the $N$-vector model.  By taking the limit as $N\to 0$, he obtained results for the self-avoiding walk.  However, there is currently no known way of making this argument rigorous, in part because $N$ is an integer and it is unclear how to rigorously allow $N\to 0$.  The meaning of the $N$-vector model when $N$ is not an integer may be thought of as a "paradoxical" object.  For more information, see Section 2.3 of The Self-Avoiding Walk by Neal Madras and Gordon Slade.
A: From what I have read, the concept of fractional order calculus still does not have a rigorous basis. It has a somehow paradoxical nature similar to that of complex numbers mentioned by the OP, and the topic is as old as the calculus itself.
Although the fractional integration can be well-defined using the Cauchy's formula, but fractional differentiation does not have a rigorous definition because of the difficulty of dealing with the initial conditions.
