Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

Background

notation: RV= random variable, $\mu=$ mean $m=$ median

Jensen's Inequality considers the relationship between the mean of a function of an RV and the function of the mean of an RV.

If $f(x)$ strictly convex:

$$\mu (f(x)) > f(\mu (x))\mathrm{\hspace{20mm}(1)}$$

Conversely if $-f(x)$ is strictly convex:

$$\mu (f(x)) < f(\mu (x))$$

An analogous property of the median has been presented (Merkle et al 2005, pdf).

Motivation

I have a nonlinear function (pdf) of positive random variables, too complex to post here, not directly pertinent to this question; I am looking for a more general answer. It is worth noting that it is, however, neither strictly concave nor convex.

In practice, I find that the function of the medians provides a much better estimate of the median of the function than does the estimate of the mean of the function from the function of the means. I am interested in learning the conditions for which this is true.

Question

Under what conditions will the function of a median be closer to the median of a function than the mean of a function is to a function of the mean?

Specifically for what types of $f(x)$ and $x$ is

$$|\mu (f(x)) - f(\mu (x))| > |m (f(x)) - f(m (x))|$$

I previously asked this on here on stats.stackexchange.com, but after not receiving an answer, was advised to post here on MO.

References

Merkle et al 2005 Jensen's inequality for medians. Statistics & Probability Letters, Volume 71, Issue 3, 1 March 2005, Pages 277-281

share|improve this question
    
Reprint available in the Statristics section of $$ $$ milanmerkle.com/… $$ $$ –  Will Jagy Nov 24 '10 at 20:45
    
I do not imagine people will have an easy time getting a printout of your own article. Could you please typeset in the function you have in mind? –  Will Jagy Nov 24 '10 at 20:55
    
@Will Thanks for your inquiry; I would typeset the function except for the fact that it is very complicated and I am interested in an answer that is independent of the particular function. –  David LeBauer Nov 24 '10 at 21:37
    
@Will, I have posted a link to a pdf of the article that should be available to all. –  David LeBauer Nov 24 '10 at 21:49
    
Do you mean to take absolute values? As in $|\mu (f(x)) - f(\mu (x))| > |m (f(x)) - f(m (x))|$? –  Robby McKilliam Dec 12 '10 at 22:04
show 1 more comment

2 Answers

up vote 4 down vote accepted

If $f\ $ is a monotonic function then the median value of $f(X)$ is the same as the function applied to the median value of $X$.

If there is no monotonicity (or approximate monotonicity) you wouldn't expect them to be even close: just think of $X$ being uniformly distributed on the interval and $f(x)=|x-1/2|$ say.

On the other hand, a reasonable condition for the mean of $f(X)$ to lie close to $f$ applied to the mean of $X$ is for $f\ $ to be close to a linear function (the same example as above shows the mean of the function being very far from the function of the mean). Of course the mean is very sensitive to extreme values of $f(X)$ whereas the median is not

In general for monotone or nearly monotone $f$, you can expect better closeness for the medians than the means. For seriously non-monotone $f$, I don't think there's anything useful you can say.

By the way, you didn't ask this, but in case you are interested in the sample mean and sample median of $f(X_1),\ldots,f(X_n)$ as compared to the true mean and median of $f(X)$, the first typically errs by about std dev$(f(X))/\sqrt n$, whereas the latter errs by about $(1/\sqrt{8n})/\rho(m(f(X)))$ where $\rho$ is the density function of $f(X)$ (which I'm assuming exists). This means you can do a numerical test to see which of these will be closer.

share|improve this answer
1  
Anthony: Nice post. I wonder what would be quantitative versions of the result you mention for functions that are close to a linear function and of the result you mention for nearly monotone functions. –  Did Jan 25 '11 at 9:45
    
@Anthony Thanks for the answer; I was hoping to find something useful to say about seriously non-monotone f, but equally interested that such something does not exist. –  David LeBauer Jan 31 '11 at 2:20
    
@David: Maybe this helps: the average of $f(X)$ takes into account all the values of $f(X)$ and takes the midpoint of them; whereas $f$ applied to the average of $X$ depends only on the value of $f$ at one point. If the function isn't monotone, the value of $f$ at one point tells you very little about the value of $f$ at all the other points. –  Anthony Quas Jan 31 '11 at 6:00
add comment

REVISED ANSWER:

Lemma 1. If $f$ is not convex, then we can find a random variable $X$ such that $\mu (f(X)) - f(\mu (X)) < m(f(X)) - f(m(X))$.

Proof. If $f$ is not convex, then, by definition, there exist two points $a \neq b$ and a $p \in (0,1)$ such that $f(pa + (1-p)b) > pf(a) + (1-p)f(b)$. If this inequality holds for $p = 1/2$ but not for $p = 3/4$, then it also holds with $p = 2/3$ and $a$ replaced by $3a/4 + b/4$. Thus, WLOG we can assume that $f(pa + (1-p)b) > pf(a) + (1-p)f(b)$ for some $a \neq b$ and $p > 1/2$. Define random variable $X$ as follows: ${\rm P}(X = a) = p$, ${\rm P}(X = b) = 1-p$. Then, $m(X) = a$ and $m(f(X)) = f(a)$, so $m(f(X)) - f(m(X)) = 0$. The lemma now follows from $\mu (f(X)) - f(\mu (X)) = pf(a) + (1-p)f(b) - f(pa + (1-p)b) < 0$.

Lemma 2. If for some $a < b < c$, $f(a) = f(c) > f(b)$, then we can find a random variable $X$ such that $\mu (f(X)) - f(\mu (X)) < m(f(X)) - f(m(X))$.

Proof. Fix $\varepsilon > 0$ sufficiently small. Define random variable $X$ as follows: ${\rm P}(X=b) = 1/2 - \varepsilon$, ${\rm P}(X=a) = (1/2 + \varepsilon)(c - b)/(c - a)$, ${\rm P}(X=c) = (1/2 + \varepsilon)(b - a)/(c - a)$. Then, $\mu(X)=m(X)=b$, and ${\rm P}[f(X) = f(b)] = 1/2 - \varepsilon$, ${\rm P}[f(X) = f(a)]= 1/2 + \varepsilon$. The lemma now follows from $\mu(f(X)) = (1/2 - \varepsilon)f(b) + (1/2 + \varepsilon)f(a) < f(a) = m(f(X))$.

From Lemmas 1 and 2, we conclude:

Corollary. The inequality $\mu (f(X)) - f(\mu (X)) \geq m(f(X)) - f(m(X))$ holds for any (integrable) random variable $X$ only if $f$ is convex and monotone.

EDIT: A somewhat trivial extension of this corollary ("if part").

Suppose that $f$ is a strictly monotone convex function, defined on an interval $I$ containing the range of an integrable rv $X$. If $m(X)$ is unique, then so is $m(f(X))$, and it holds $\mu (f(X)) - f(\mu (X)) \geq m(f(X)) - f(m(X)) = 0$.

Proof. By Jensen's inequality, $\mu (f(X)) - f(\mu (X)) \geq 0$. So, it remains to show that $m(f(X)) = f(m(X))$. Suppose that $\tilde m \in f(I)$ is a median of $f(X)$. Then, by definition, ${\rm P}(f(X) \leq \tilde m) \geq 1/2$ and ${\rm P}(f(X) \geq \tilde m) \geq 1/2$. Taking inverses shows that $f^{-1}(\tilde m)$ is a median of $X$. Thus, $f^{-1}(\tilde m) = m(X)$, and the assertion follows.

share|improve this answer
    
My answer only concerns the question "for what types of $f$ is $\mu (f(x)) - f(\mu (x)) \geq m (f(x)) - f(m (x))$ for any (integrable) random variable $X$." –  Shai Covo Nov 28 '10 at 15:35
    
Also, my answer corresponds to the original question (see Robby McKilliam's comment above). –  Shai Covo Dec 14 '10 at 0:15
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.