18
$\begingroup$

Background

notation: RV= random variable, $\mu=$ mean $m=$ median

Jensen's Inequality considers the relationship between the mean of a function of an RV and the function of the mean of an RV.

If $f(x)$ strictly convex:

$$\mu (f(x)) > f(\mu (x))\mathrm{\hspace{20mm}(1)}$$

Conversely if $-f(x)$ is strictly convex:

$$\mu (f(x)) < f(\mu (x))$$

An analogous property of the median has been presented (Merkle et al 2005, pdf).

Motivation

I have a nonlinear function (pdf) of positive random variables, too complex to post here, not directly pertinent to this question; I am looking for a more general answer. It is worth noting that it is, however, neither strictly concave nor convex.

In practice, I find that the function of the medians provides a much better estimate of the median of the function than does the estimate of the mean of the function from the function of the means. I am interested in learning the conditions for which this is true.

Question

Under what conditions will the function of a median be closer to the median of a function than the mean of a function is to a function of the mean?

Specifically for what types of $f(x)$ and $x$ is

$$|\mu (f(x)) - f(\mu (x))| > |m (f(x)) - f(m (x))|$$

I previously asked this on here on stats.stackexchange.com, but after not receiving an answer, was advised to post here on MO.

References

Merkle et al 2005 Jensen's inequality for medians. Statistics & Probability Letters, Volume 71, Issue 3, 1 March 2005, Pages 277-281

$\endgroup$
6
  • $\begingroup$ Reprint available in the Statristics section of $$ $$ milanmerkle.com/… $$ $$ $\endgroup$
    – Will Jagy
    Nov 24, 2010 at 20:45
  • $\begingroup$ I do not imagine people will have an easy time getting a printout of your own article. Could you please typeset in the function you have in mind? $\endgroup$
    – Will Jagy
    Nov 24, 2010 at 20:55
  • $\begingroup$ @Will Thanks for your inquiry; I would typeset the function except for the fact that it is very complicated and I am interested in an answer that is independent of the particular function. $\endgroup$ Nov 24, 2010 at 21:37
  • $\begingroup$ @Will, I have posted a link to a pdf of the article that should be available to all. $\endgroup$ Nov 24, 2010 at 21:49
  • $\begingroup$ Do you mean to take absolute values? As in $|\mu (f(x)) - f(\mu (x))| > |m (f(x)) - f(m (x))|$? $\endgroup$ Dec 12, 2010 at 22:04

2 Answers 2

4
$\begingroup$

If $f\ $ is a monotonic function then the median value of $f(X)$ is the same as the function applied to the median value of $X$.

If there is no monotonicity (or approximate monotonicity) you wouldn't expect them to be even close: just think of $X$ being uniformly distributed on the interval and $f(x)=|x-1/2|$ say.

On the other hand, a reasonable condition for the mean of $f(X)$ to lie close to $f$ applied to the mean of $X$ is for $f\ $ to be close to a linear function (the same example as above shows the mean of the function being very far from the function of the mean). Of course the mean is very sensitive to extreme values of $f(X)$ whereas the median is not

In general for monotone or nearly monotone $f$, you can expect better closeness for the medians than the means. For seriously non-monotone $f$, I don't think there's anything useful you can say.

By the way, you didn't ask this, but in case you are interested in the sample mean and sample median of $f(X_1),\ldots,f(X_n)$ as compared to the true mean and median of $f(X)$, the first typically errs by about std dev$(f(X))/\sqrt n$, whereas the latter errs by about $(1/\sqrt{8n})/\rho(m(f(X)))$ where $\rho$ is the density function of $f(X)$ (which I'm assuming exists). This means you can do a numerical test to see which of these will be closer.

$\endgroup$
3
  • 1
    $\begingroup$ Anthony: Nice post. I wonder what would be quantitative versions of the result you mention for functions that are close to a linear function and of the result you mention for nearly monotone functions. $\endgroup$
    – Did
    Jan 25, 2011 at 9:45
  • $\begingroup$ @Anthony Thanks for the answer; I was hoping to find something useful to say about seriously non-monotone f, but equally interested that such something does not exist. $\endgroup$ Jan 31, 2011 at 2:20
  • $\begingroup$ @David: Maybe this helps: the average of $f(X)$ takes into account all the values of $f(X)$ and takes the midpoint of them; whereas $f$ applied to the average of $X$ depends only on the value of $f$ at one point. If the function isn't monotone, the value of $f$ at one point tells you very little about the value of $f$ at all the other points. $\endgroup$ Jan 31, 2011 at 6:00
3
$\begingroup$

REVISED ANSWER:

Lemma 1. If $f$ is not convex, then we can find a random variable $X$ such that $\mu (f(X)) - f(\mu (X)) < m(f(X)) - f(m(X))$.

Proof. If $f$ is not convex, then, by definition, there exist two points $a \neq b$ and a $p \in (0,1)$ such that $f(pa + (1-p)b) > pf(a) + (1-p)f(b)$. If this inequality holds for $p = 1/2$ but not for $p = 3/4$, then it also holds with $p = 2/3$ and $a$ replaced by $3a/4 + b/4$. Thus, WLOG we can assume that $f(pa + (1-p)b) > pf(a) + (1-p)f(b)$ for some $a \neq b$ and $p > 1/2$. Define random variable $X$ as follows: ${\rm P}(X = a) = p$, ${\rm P}(X = b) = 1-p$. Then, $m(X) = a$ and $m(f(X)) = f(a)$, so $m(f(X)) - f(m(X)) = 0$. The lemma now follows from $\mu (f(X)) - f(\mu (X)) = pf(a) + (1-p)f(b) - f(pa + (1-p)b) < 0$.

Lemma 2. If for some $a < b < c$, $f(a) = f(c) > f(b)$, then we can find a random variable $X$ such that $\mu (f(X)) - f(\mu (X)) < m(f(X)) - f(m(X))$.

Proof. Fix $\varepsilon > 0$ sufficiently small. Define random variable $X$ as follows: ${\rm P}(X=b) = 1/2 - \varepsilon$, ${\rm P}(X=a) = (1/2 + \varepsilon)(c - b)/(c - a)$, ${\rm P}(X=c) = (1/2 + \varepsilon)(b - a)/(c - a)$. Then, $\mu(X)=m(X)=b$, and ${\rm P}[f(X) = f(b)] = 1/2 - \varepsilon$, ${\rm P}[f(X) = f(a)]= 1/2 + \varepsilon$. The lemma now follows from $\mu(f(X)) = (1/2 - \varepsilon)f(b) + (1/2 + \varepsilon)f(a) < f(a) = m(f(X))$.

From Lemmas 1 and 2, we conclude:

Corollary. The inequality $\mu (f(X)) - f(\mu (X)) \geq m(f(X)) - f(m(X))$ holds for any (integrable) random variable $X$ only if $f$ is convex and monotone.

EDIT: A somewhat trivial extension of this corollary ("if part").

Suppose that $f$ is a strictly monotone convex function, defined on an interval $I$ containing the range of an integrable rv $X$. If $m(X)$ is unique, then so is $m(f(X))$, and it holds $\mu (f(X)) - f(\mu (X)) \geq m(f(X)) - f(m(X)) = 0$.

Proof. By Jensen's inequality, $\mu (f(X)) - f(\mu (X)) \geq 0$. So, it remains to show that $m(f(X)) = f(m(X))$. Suppose that $\tilde m \in f(I)$ is a median of $f(X)$. Then, by definition, ${\rm P}(f(X) \leq \tilde m) \geq 1/2$ and ${\rm P}(f(X) \geq \tilde m) \geq 1/2$. Taking inverses shows that $f^{-1}(\tilde m)$ is a median of $X$. Thus, $f^{-1}(\tilde m) = m(X)$, and the assertion follows.

$\endgroup$
2
  • $\begingroup$ My answer only concerns the question "for what types of $f$ is $\mu (f(x)) - f(\mu (x)) \geq m (f(x)) - f(m (x))$ for any (integrable) random variable $X$." $\endgroup$
    – Shai Covo
    Nov 28, 2010 at 15:35
  • $\begingroup$ Also, my answer corresponds to the original question (see Robby McKilliam's comment above). $\endgroup$
    – Shai Covo
    Dec 14, 2010 at 0:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.