I'm not sure if this is a soft question, and whether it may be too broad or, on the contrary, too localized. Well, in Mathematics the concept of "infinitesimal" has been of extreme importance for centuries.
The present mathematical tecnology allows one, according to the context, to formalize this notion in several ways:
- Differential geometry. The classical epsilon-delta formalism of limits in elementary analysis leads to the concept of first-order (or $n$-th order) approximation in Calculus, hence to many standard notions in differential geometry: the differential of a map between smooth manifolds, jets of a map, tangent vectors, differential forms, and Riemannian metrics.
- Algebraic geometry. The lack of a useful notion of convergence of sequences due to the coarseness of Zariski topology prevents us from using epsilon-delta arguments to define "infinitesimals". But then one retains the notion of first-order (or $n$-th order) "approximation" in a more formal way, e.g. by means of universal properties of modules and derivations (Kahler differentials...), and that of "infinitesimal space" e.g. by means of local Artinian $\Bbbk$-algebras, and of "infinitesimal neighbourhood" e.g. by completion of local rings, formal schemes etc. (But after all the algebro geometric perspective is just a high brow way of doing the formal derivative of polynomials, which coincides with the "topological one" once we work over $\mathbb{R}$ or $\mathbb{C}$)
- Synthetic differential geometry. More recently some mathematicians are exploring the realm of synthetic differential geometry, of which I know nothing except that it kind of unifies the perspectives of the previous two approaches and uses relevant amounts of category theory (please correct me if I'm not being correct).
- Non-standard analysis. The concept of infinitesimal element is of course fundamental in non-standard analysis, where the field $\mathbb{R}$ and the use of epsilon-delta arguments is replaced by the introduction of the field ${}^* \mathbb{R}$ of hyperreal numbers, which hosts several hierarchies of infinitesimals (as well as "infinite elements").
- Noncommutative geometry. According to A.Connes (in his book Noncommutative Geometry there's a paragraph on a "Quantized calculus") given an infinite dimensional separable Hilbert space $\mathcal{H}$ and a certain operator $F$ on it, compact operators on $\mathcal{H}$ with characteristic values such that $\mu_n=O(n^{-\alpha})$, $n\to\infty$, can be interpreted as "infinitesimals of order $\alpha$", while the differential of a "complex variable" (read "operator on $\mathcal{H}$") $f$ is just defined to be the commutator $[F,f]$.
At the risk of seeing my question closed as too vague ("not a real question"), it would be my curiosity to know:
Is there a theory that encompasses all the above instances of "infinitesimals" within a unique formal picture? Or, on the contrary, are some of the above notions of infinitesimals inherently specific to their field and embody formalizations of different heuristic notions? [A situation as in the second question occurs with the notion of "infinity": it seems to me there's almost no deep relation between the infinity as in $\lim_{n\to\infty}$ and the infinite cardinals of Cantor]