I don't know the literature, but it maybe helpful to understand exactly what this notion of differentiability gains for us.

First, **unlike** the Sobolev notions, this does not handle functions which belong in Holder classes. For example, based on the definition the absolute value function is **not** $L^p$ differentiable for any $p$, but it is certainly (locally) in the Sobolev class $W^{1,p}$ for **every** $p\in [1,\infty]$.

In fact, whenever $f$ is a function such that the one sided derivatives from the left and from the right independently exist, but do not agree, such a function is **not** $L^p$ differentiable in the sense defined in the paper.

So what does $L^p$ differentiability gain for us? It gains when your function fails to be differentiable near $x$ due to it being **oscillatory** in a certain way. An example:

Let the set $A = \cup_{k = 10}^\infty [ 1/k - 2^{-k}, 1/k + 2^{-k}]$, and let $f$ be the indicator function of $A$. This function is clearly not differentiable at $x = 0$. However, since

$$ \int_{-h}^h |f(s)|^p ~ds \leq \sum_{k = \lfloor 1/h \rfloor }^\infty 2^{1-k} = 2^{2-\lfloor 1/h \rfloor } $$

we see that for any $p\in [1,\infty)$, $f$ is $L^p$ differentiable at the origin with derivative $0$.

As to why measuring things with respect to $L^p$ means can be useful: there is a big hint in the paper of Calderon and Zygmund, concerning elliptic PDEs. They wrote [emphases mine]:

It seems that the notion of differentiability which is most suited to the treatment of the problems that concern us, is not the classical one. It appears that it is more convenient to estimate the remainder of the Taylor series *in the mean with various exponents*. This type of differentiability is **much more stable** ...

If you are familiar with some harmonic analysis, what this is screaming out is that a lot of analytic estimates (Sobolev regularity for elliptic PDEs, singular integrals, etc.) work generally for functions measured on the $L^p$ scale, but often fail (just by a little) at $L^\infty$ (and sometimes $L^1$).

You can of course ask why did Calderon and Zygmund not use the Sobolev class: the key is that they want to understand **pointwise** estimates of differentiability. As mentioned above, Sobolev classes are not so sensitive to pointwise properties.

Local properties of solutions of elliptic partial differential equationsby Calderón/Zygmund (1961), but all the citations I've seen are for both first order $L^p$ derivatives and higher order $L^p$ derivatives, leaving open the possibility that the first order $L^p$ derivative might have appeared earlier. Some motivation can be found in Ash's 2014 paperRemarks on various generalized derivatives. The top of p. 9 (.pdf file page number) Ash writes:(continued)$\endgroup$ – Dave L Renfro May 7 at 8:33