When is the function of a median closer to the median of the function than the mean of the function is to the function of the mean? - MathOverflow

When is the function of a median closer to the median of the function than the mean of the function is to the function of the mean?

2010-11-24T19:40:46Z

Background

notation: RV= random variable, $\mu=$ mean $m=$ median

Jensen's Inequality considers the relationship between the mean of a function of an RV and the function of the mean of an RV.

If $f(x)$ strictly convex:

$$\mu (f(x)) > f(\mu (x))\mathrm{\hspace{20mm}(1)}$$

Conversely if $-f(x)$ is strictly convex:

$$\mu (f(x)) < f(\mu (x))$$

An analogous property of the median has been presented (Merkle et al 2005, pdf).

Motivation

I have a nonlinear function (pdf) of positive random variables, too complex to post here, not directly pertinent to this question; I am looking for a more general answer. It is worth noting that it is, however, neither strictly concave nor convex.

In practice, I find that the function of the medians provides a much better estimate of the median of the function than does the estimate of the mean of the function from the function of the means. I am interested in learning the conditions for which this is true.

Question

Under what conditions will the function of a median be closer to the median of a function than the mean of a function is to a function of the mean?

Specifically for what types of $f(x)$ and $x$ is

$$|\mu (f(x)) - f(\mu (x))| > |m (f(x)) - f(m (x))|$$

I previously asked this on here on stats.stackexchange.com, but after not receiving an answer, was advised to post here on MO.

References

Merkle et al 2005 Jensen's inequality for medians. Statistics & Probability Letters, Volume 71, Issue 3, 1 March 2005, Pages 277-281

Answer by Shai Covo for When is the function of a median closer to the median of the function than the mean of the function is to the function of the mean?

2010-11-25T14:14:25Z

REVISED ANSWER:

Lemma 1. If $f$ is not convex, then we can find a random variable $X$ such that $\mu (f(X)) - f(\mu (X)) < m(f(X)) - f(m(X))$ .

Proof. If $f$ is not convex, then, by definition, there exist two points $a \neq b$ and a $p \in (0,1)$ such that $f(pa + (1-p)b) > pf(a) + (1-p)f(b)$ . If this inequality holds for $p = 1/2$ but not for $p = 3/4$ , then it also holds with $p = 2/3$ and $a$ replaced by $3a/4 + b/4$ . Thus, WLOG we can assume that $f(pa + (1-p)b) > pf(a) + (1-p)f(b)$ for some $a \neq b$ and $p > 1/2$ . Define random variable $X$ as follows: ${\rm P}(X = a) = p$ , ${\rm P}(X = b) = 1-p$ . Then, $m(X) = a$ and $m(f(X)) = f(a)$ , so $m(f(X)) - f(m(X)) = 0$ . The lemma now follows from $\mu (f(X)) - f(\mu (X)) = pf(a) + (1-p)f(b) - f(pa + (1-p)b) < 0$ .

Lemma 2. If for some $a < b < c$ , $f(a) = f(c) > f(b)$ , then we can find a random variable $X$ such that $\mu (f(X)) - f(\mu (X)) < m(f(X)) - f(m(X))$ .

Proof. Fix $\varepsilon > 0$ sufficiently small. Define random variable $X$ as follows: ${\rm P}(X=b) = 1/2 - \varepsilon$ , ${\rm P}(X=a) = (1/2 + \varepsilon)(c - b)/(c - a)$ , ${\rm P}(X=c) = (1/2 + \varepsilon)(b - a)/(c - a)$ . Then, $\mu(X)=m(X)=b$ , and ${\rm P}[f(X) = f(b)] = 1/2 - \varepsilon$ , ${\rm P}[f(X) = f(a)]= 1/2 + \varepsilon$ . The lemma now follows from $\mu(f(X)) = (1/2 - \varepsilon)f(b) + (1/2 + \varepsilon)f(a) < f(a) = m(f(X))$ .

From Lemmas 1 and 2, we conclude:

Corollary. The inequality $\mu (f(X)) - f(\mu (X)) \geq m(f(X)) - f(m(X))$ holds for any (integrable) random variable $X$ only if $f$ is convex and monotone.

EDIT: A somewhat trivial extension of this corollary ("if part").

Suppose that $f$ is a strictly monotone convex function, defined on an interval $I$ containing the range of an integrable rv $X$ . If $m(X)$ is unique, then so is $m(f(X))$ , and it holds $\mu (f(X)) - f(\mu (X)) \geq m(f(X)) - f(m(X)) = 0$ .

Proof. By Jensen's inequality, $\mu (f(X)) - f(\mu (X)) \geq 0$ . So, it remains to show that $m(f(X)) = f(m(X))$ . Suppose that $\tilde m \in f(I)$ is a median of $f(X)$ . Then, by definition, ${\rm P}(f(X) \leq \tilde m) \geq 1/2$ and ${\rm P}(f(X) \geq \tilde m) \geq 1/2$ . Taking inverses shows that $f^{-1}(\tilde m)$ is a median of $X$ . Thus, $f^{-1}(\tilde m) = m(X)$ , and the assertion follows.

Answer by Anthony Quas for When is the function of a median closer to the median of the function than the mean of the function is to the function of the mean?

2011-01-25T09:18:00Z

If $f\ $ is a monotonic function then the median value of $f(X)$ is the same as the function applied to the median value of $X$.

If there is no monotonicity (or approximate monotonicity) you wouldn't expect them to be even close: just think of $X$ being uniformly distributed on the interval and $f(x)=|x-1/2|$ say.

On the other hand, a reasonable condition for the mean of $f(X)$ to lie close to $f$ applied to the mean of $X$ is for $f\ $ to be close to a linear function (the same example as above shows the mean of the function being very far from the function of the mean). Of course the mean is very sensitive to extreme values of $f(X)$ whereas the median is not

In general for monotone or nearly monotone $f$, you can expect better closeness for the medians than the means. For seriously non-monotone $f$, I don't think there's anything useful you can say.

By the way, you didn't ask this, but in case you are interested in the sample mean and sample median of $f(X_1),\ldots,f(X_n)$ as compared to the true mean and median of $f(X)$, the first typically errs by about std dev$(f(X))/\sqrt n$, whereas the latter errs by about $(1/\sqrt{8n})/\rho(m(f(X)))$ where $\rho$ is the density function of $f(X)$ (which I'm assuming exists). This means you can do a numerical test to see which of these will be closer.