Personally, if I was entering this subject blind I would feel cheated if not shown the extensive pure mathematical power of the fractional derivative. Being that it is more useful than just being used to solve differential equations or physical problems.

The first thing is to look at Cauchy's integral formula which is most aptly

$$\int_a^x \int_a^{x_{n-1}}...\int_a^{x_1}f(x_0)\,dx_0dx_1dx_2...dx_{n-1} = \frac{1}{n-1!}\int_a^x f(y)y^{n-1}\,dy$$

which is a strikingly powerful equation. The natural generalisation arises by considering the operator $I_a f = \int_a^x f(y)\,dy$ and simply writing
$$I_a^n = I_a ... (n\,times)...I_a f = \frac{1}{n-1!}\int_a^x f(y)y^{n-1}\,dy$$

where a natural conclusion is to define

$$I_{a}^z f = \frac{1}{\Gamma(z)}\int_a^x f(y)y^{z-1}\,dy$$

which through no obvious or simple method

$$I_a^{z_0}I_a^{z_1} = I_a^{z_0 + z_1}$$

This gives not only one iterated "fractional" integral but infinitely many for each $a$. The perspective result, or canonical fact, is that each fractional integral satisfies

$$I_a^z (x-a)^r = \frac{\Gamma(r+1)}{\Gamma(r+z+1)}(x-a)^{r+z}$$

and $I_a (x-b)^r$ when $b \neq a$ is defined using a binomial expansion.

Defining $\frac{d}{dx}_a^z = I_a^{-z}$ for $\Re(z) < 0$ and $\frac{d}{dx}_a^z = \frac{d}{dx}^n I_a^{n-z}$ for $\Re(z) < n$ we arrive at a fractional derivative.

This seemingly convenient and beautiful expression gives us something rather ugly though. Since $\frac{d}{dx} e^x = e^x$ we would like $\frac{d}{dx}^z e^x = e^x$, but this is not so. By uniform convergence and all that jazz

$$\frac{d}{dx}_a^z e^x = \sum_{n=0}^\infty \frac{x^{n-z}}{\Gamma(n+1-z)}$$

which is not $e^x$.

Therefore another fractional derivative is required. Taking $a = -\infty$ then we arrive at the commonly called "exponential differintegral" which can be written

$$\frac{d}{dx}^{-z} f(x) = \frac{1}{\Gamma(z)}\int_0^\infty f(x-y)y^{z-1}\,dx$$ defined for $f$ satisfying specific decay conditions at negative infinity. As one can see this fractional derivative fixes $e^x$ but diverges for any polynomial.

Now we can generalize this even further!

Consider $f(w)$ entire on $\mathbb{C}$, and for convenience assume $f(w)w \to 0$ as $w \to \infty$ when $|\arg(w)| < \kappa$ and call this space of function $D_\kappa$

Then we have the disastrously large formula

$$\frac{d^z}{dw^z} f(w) = \frac{e^{i\theta z}}{\Gamma(-z)}\Big{(}\sum_{n=0}^\infty f^{(n)}(w)\frac{(-e^{i\theta})^n}{n!(n-z)} + \int_1^\infty f(w-e^{i\theta}y)y^{-z-1}\,dy\Big{)}$$

which holds for all $|\theta| < \kappa$ and $\Re(z) > -1$.

Now some people would rashly think what is the point of this? Some interesting things happen in this scenario, firstly the differintegral can be thought of as a modified Mellin transform. Giving us things like Ramanujan's master theorem in a slicker notation. It further emphasizes that this operator arises in a very natural sense (the Mellin transform being prominent in many areas of mathematics). It says $\frac{d^z}{dw^z}$ for $\Re(z) > 0$ takes $D_\kappa$ to itself. So we have a semigroup $\{\frac{d^z}{dw^z} | \Re(z) > 0\}$ acting on $D_\kappa$.

Furthermore, when looking at the fourier transform definition of a fractional derivative, it is in fact this clunky looking exponential derivative that's really pulling the strings. Where it may seem cleaner in Fourier transforms, it is much more general in its Mellin form.

All in all it is quite a mysterious object, and is underused in my opinion.