0
$\begingroup$

Originally posted on Math.StackExchange, here, but I was advised to post it on MathOverflow as it is a research question. Now two final, great answers have been posted, see on Math.StackExchange.

I am interested in the derivative of a function defined on a subset $S$ of $[0, 1]$. The subset in question is dense in $[0, 1]$ but has Lebesgue measure zero. My actual question can be found at the bottom of this post.

There has been a few questions on the subject, but none leading to something interesting as far as I am concerned. See here, here (concept of arithmetic derivative) and also here (Minkowski's question mark function, related to the material discussed here.)

Generally these discussions lead to some kind of non-sense math. Here it is the opposite. I have a framework that does work as far as applications and computations are concerned, but I have a hard time putting it into some sound mathematical framework. It would have to be some kind of non-standard calculus.

Perhaps the simplest example (though I have plenty of other similar cases) is as follows. Let $Z$ be a random variable defined as follows: $$Z = \sum_{k=1}^\infty \frac{X_k}{2^k}$$

where the $X_k$'s are identically and independently distributed with a Bernouilli$(p)$ distribution. Thus $P(X_k = 1) = p$ and $P(X_k = 0) = 1-p$. Here $0 < p < 1$. In short, the $X_k$'s are the binary digits of the random number $Z$.

There are two cases.

Case $p=\frac{1}{2}$

In this case, $Z$ has a uniform distribution on $S$, where $S$ is the set of normal numbers in $[0, 1]$. It is known that $S$ has Lebesgue measure $1$, and that $S$ is dense in $[0, 1]$. Yet it is full of holes (no rational number is is a normal number due to their periodicity, thus the $X_k$'s are not independent for rational numbers.)

This is the simplest case. One might wonder if the density $f_Z$ (the derivative of the distribution $F_Z$) exists. Yet $f_Z(z) = 1$ if $z \in S$ works perfectly well for all purposes. It can easily be extended to $f_Z(z) = 1$ if $z \in [0, 1]$. Let us denote the extended function as $\tilde{f}_Z$. You can compute all the moments using the extended $\tilde{f}_Z$ and get the right answer. If $s$ is a positive real number, then $$E(Z^s) = \int_0^1 z^s \tilde{f}_Z(z) dz = \frac{1}{s+1}.$$

You could argue that $\tilde{f}_Z$ (and thus $f_Z$) can be obtained by inverting the above functional equation, using some kind of Laplace transform. So we can bypass the concept of derivative entirely, it seems.

Case $p\neq \frac{1}{2}$

Now we are dealing with a hard nut to crack, and a wildly chaotic system: $Z$'s support domain is a set $S'$ that is a subset of non-normal numbers in $[0, 1]$. This set $S'$ has now Lebesgue measure zero, yet it is dense in $[0, 1]$. For the distribution, it is not a problem: even discrete random variables have a distribution $F_Z$ defined for all positive real numbers: $F_Z(z) = P(Z \leq z)$.

The issue is with the density $f_Z = dF_Z/dz$. It sounds either it should be zero everywhere or not exist. My guess is that you might be able to define a new, workable concept of density. In the neighborhood of every point $z \in S'$, it looks like $g(z,h) = (F_Z(z+h) - F_Z(z))/h$ oscillates infinitely many times with no limit as $h\rightarrow 0$, yet these oscillations are bounded most of the time, perhaps leading to the fact that averaging $g(z, h)$ around $h = 0$, using smaller and smaller values of $h$, could provide a sound definition for the density $f_Z$.

Again, despite the chaotic nature of the system (see how the the would-be density could potentially look like in the picture below) all the following quantities exist and can be computed exactly and then confirmed by empirical evidence, even though the integrals below may not make sense:

$$E(Z) = \int_{0}^{1} z f_Z(z) dz = p \\ E(Z^2) = \int_{0}^{1} z^2 f_Z(z) dz =\frac{p}{3}(1+2p)\\ E(Z^3) = \int_{0}^{1} z^3 f_Z(z) dz =\frac{p}{7}(1+4p+2p^2)\\ E(Z^4) = \int_{0}^{1} z^4 f_Z(z) dz =\frac{p}{105}(7+46p + 44p^2+8p^3) $$ Indeed, a general formula for $E(Z^s) = \int_0^1 z^s f_Z(z)dz$ is available for $s \geq 0$, defined by the following functional equation (see here): $$E(Z^s) = \frac{p}{2^s-1+p}\cdot E((1+Z)^s) .$$

In other words, we would have, under some appropriate calculus theory with a sound definition of integral and derivative: $$\int_{S'}z^s f_Z(z)dz = \frac{p}{2^s-1+p}\cdot\int_{S'}(1+z)^s f_Z(z) dz .$$

Here is how the density $f_Z$, if properly defined, could look like for $p=0.75$ (see here and here):

enter image description here

Below is the empirical percentile distribution, for this particular $Z$:

enter image description here

Other related problems

If instead, we consider the model $Z = X_1 + X_1 X_2 + X_1 X_2 X_3 + \cdots$ with $X$ Bernouilli$(p)$, then $Z$ has a geometric distribution of parameter $1-p$, see section 2.2 in this article. This system is also equivalent to the binary numeration system discussed so far: see section 5 in the same article. It results in $Z$ having a standard, well-known discrete distribution. But if this time $P(X=-0.5) = 0.5 = P(X=0.5)$ then $Z$ is uniform on a subset $S$ of $[-1, 1]$, with $S$ also full of holes.

Here is another interesting model:

$$Z=\sqrt{X_1+\sqrt{X_2+\sqrt{X_3+\cdots}}}$$

The distribution for $X$ is as follows: $$P(X=0) = 1/2, P(X=1) = 1 / (1 + \sqrt{5}), P(X=2) = (3 - \sqrt{5}) / 4.$$

This corresponds to a different numeration system, and the choice for $X$'s distribution is not arbitrary, see here. Normal numbers in that system are very different from normal numbers in the binary numeration system. The successive digits have a very specific auto-correlation structure, and the digits $0, 1, 2$ are not evenly distributed for normal numbers in that system. It is clear that if we assume that the $X_k$'s are i.i.d, then $Z$ is not a normal number in that system. Yet we get a very good approximation for $F_Z$, much smoother (at least visually) than in the binary numeration system investigated earlier, with $p\neq \frac{1}{2}$. In particular, $F_Z$ is very well approximated by a log function, see chart below.

enter image description here

Here the blue line is the empirical distribution for $Z$, the red line is the log approximation. And below is a spectacular chart, featuring the approximation error $\epsilon(z) = F_Z(z) -\log_2(z)$. It's a fractal! (source: see section 2.2 in this article). In short, it is no more differentiable than a Brownian motion, and technically, the derivative $f_Z$ does not exist. Yet all moments of $Z$ can be computed exactly from the functional equation attached to that system ($F_{Z^2}=F_{X+Z}$) and confirmed empirically. Even though the distribution looks smooth to the naked eye, we are dealing here with a very chaotic system in disguise. Again we need non-standard calculus to handle the density, whose support is a dense set of Lebesgue measure zero in $[1, 2]$.

Since fractals are nowhere differentiable, $f_Z$ does not exist. Yet one could imagine a "density" that would look like $f_Z=\frac{1}{z}$ for $z\in [1,2]$.

enter image description here

My question

Is there an existing theory to handle this type of density-like-substance?

$\endgroup$
5
  • 2
    $\begingroup$ The distribution of $Z$ is singular with respect to the Lebesgue measure, so no such $f_Z$ exists. It looks like measure theory is what you are after, but I fail to fully understand your question. The $g(z,h)/h$ converges to zero at Lebesgue almost every point of $[0,1]$ and to infinity at almost every point of $[0,1]$ with respect to the distribution of $Z$ (by the Radon–Nikodym theorem and the Lebesgue differentiation theorem). $\endgroup$ Dec 7, 2019 at 8:52
  • 1
    $\begingroup$ Please edit in a link to the m.se question, and edit a link there to this question. $\endgroup$ Dec 7, 2019 at 10:49
  • $\begingroup$ I did it, thank you. $\endgroup$ Dec 7, 2019 at 16:09
  • $\begingroup$ @MateusZ: Maybe there is just not solution. The density does not exist of course (unless you smooth out the distribution, a possibility), maybe it is just an artificial, symbolic construct but un-computable. An artificial construct that may play some role in establishing some formulas though. $\endgroup$ Dec 7, 2019 at 16:23
  • $\begingroup$ Or to put it differently, what would be the densities closest to satisfying the functional equation? In my last example, $f_Z(z)=1/z$ on $[1,2]$ is a good candidate. No solution (for the density) exists, but maybe we can find a non-solution that minimizes some error criterion. In short, an approximation to an object that does not exist, yet doing a good job for practical purposes. $\endgroup$ Dec 7, 2019 at 21:11

0

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.