For expectation (mean), there are many useful properties such as Linearity of Expectation:
- $\mathbb{E}[X+Y]=\mathbb{E}[X]+\mathbb{E}[Y]$
- $\mathbb{E}[\alpha X]=\alpha\mathbb{E}[X]$
(The 2 equations above can be proved either by definition or convolution)
If we want to extract statistics information such as median and mean through data sampled from a distribution, we can simply minimize the $L_1$ and $L_2$ loss respectively, as mentioned here.
To generalize these statistics further, median can be generalized into quantiles, while mean can be generalized into expectiles. We can also extract them through assymetric $L_1$ and $L_2$ losses, as mentioned here.
These statistics are really useful in state-of-the-art DRL methods, such as QR-DQN, IQN, FQF...
(Expectiles are mostly used in Economics, so they are less known comparing to quantiles.)
The $\tau\in(0,1)$ expectile with value $t$ of a random variable $X$ with c.d.f. $F(x)$ is defined as:
- $(1-\tau)\int^t_{-\infty}(t-x)\mathrm{d}F(x)=\tau\int^\infty_t(x-t)\mathrm{d}F(x)$
shown in Ehm et al., 2015
- $\int^t_{-\infty}|t-x|\mathrm{d}F(x)=\tau\int^\infty_{-\infty}|x-t|\mathrm{d}F(x)$
shown in Gu & Zou, 2016
- $t-\mathbb{E}[X]=\frac{2\tau-1}{1-\tau}\int^\infty_t(x-t)\mathrm{d}F(x)$
shown in Newey & Powell, 1987
Where $\mathbb{E}[X]=\int^\infty_{-\infty}xf(x)\mathrm{d}x=\int^\infty_{-\infty}x\mathrm{d}F(x)$
(The 3 equations above are all mathematically equivalent.)
Here are the visualization of the mean and expectiles in terms of the c.d.f. of a random variable:
I'm wondering that do Linearity of Expectiles exist? (Like in Linearity of Expectation?)
That is, does the following properties hold for 2 random variables $X$ and $Y$, with independent $\tau$-expectile value $t_X$, $t_Y$ respectively, and the sum $X+Y$ with $\tau$-expectile value $t_{X+Y}$:
- $t_{X+Y}=t_X+t_Y$
- $t_{\alpha X}=\alpha t_X$
(The 2 equations above hold when $\tau=0.5$, where $t_{X+Y}=\mathbb{E}[X+Y]$)
If the Linearity of Expectiles holds for all $\tau\in(0,1)$, the expectiles will become more mathematical friendly when calculating the statistics of the sum of multiple random variables. If the Linearity of Expectiles does not hold, in what condition does the linearity hold?
(Do $X$ and $Y$ need to be independent like for the Linearity of Variance? Or do $X$ and $Y$ need to have certain probability distributions like for the Linearity of Median?)