This is not what I would call "nonstandard", but you may be interested in surreal numbers. They were originally developed for combinatoric game theory but they allow somewhat unique circumstances in treating infinity and infinitesimals; like the paper below explains, using surreal numbers you can discuss "the square root of infinity" without it being complete nonsense.

Link to fairly good introduction

This is not what I would call "nonstandard", but you may be interested in surreal numbers. They were originally developed for combinatoric game theory but they allow somewhat unique circumstances in treating infinity and infinitesimals; like the paper below explains, using surreal numbers you can discuss "the square root of infinity" without it being complete nonsense.

Link to fairly good introduction

Here's an brief summary:

1. A surreal number is a pair of sets of surreal numbers L (the "left set") and R (the "right set") such that no member of R is less than or equal to any member of L. Traditionally the numbers are written { L | R }.

2. Given surreal numbers x and y, $x \leq y$ if and only if y is less than or equal to no member of x’s left set, and no member of y’s right set is less than or equal to x.

3. We define the surreal number {|} to be 0.

These definitions (along with logical ones for addition, subtraction, etc.) spin out an entire number system, and it ends up that one version of $\epsilon$ is {0 | 1, 1/2, 1/4, 1/8, ...} and one version of $\omega$ is {$\mathbb{Z}$ | 0}.

While having infinity and infinitesimals as actual numbers in the system sounds like a good deal, it makes integration or differentation hard (I don't believe anyone has yet found a method).