Return to Question

Bounty Ended with Robert Israel's answer chosen by Gil Kalai

occurred Jun 16, 2012 at 18:52

Bounty Started worth 50 reputation by Gil Kalai

occurred Jun 9, 2012 at 20:10

edited tags

Link

edited Jun 9, 2012 at 20:10

24.7k
38
235
327

pr.probability co.combinatorics probability-distributions

added 1140 characters in body

Source Link

edited Jun 9, 2012 at 18:42

24.7k
38
235
327

Start with a continuous probability distribution given by a density function f(x). Let X be a real random variable whose distribution is given by the probability distribution.

I would like to ask about the following specific way to describe the distribution in terms of the variance of tails.

Given a number t between 0 and 1 let s be such that the probability that $X\ge s$ equals t. (In other words, $\int_s^{\infty}f(x)=t$.) Let $W_X(t)$ be the variance of $X$ conditioned on $X\ge s$.

The function $W_X(t)$ seems an interesting way to describe the original probability distribution, and its tail behavior. I wonder if there is an explicit useful way to move from $W$ back to $f$; Can you give a dictionary between the tail behavior of $f$ and of that of $W$; And an explicit description of $W$ for some "famous" probability distributions. (E.g. the normal distribution, and the Tracy Widom distribution.) And of course if this kind of "transform" is considered in the literature.

UPDATE: Many thanks to Robert and Brendan to their answers. Robert's asymptotic expansion seem amazing. I will be happy to understand the asymptotics as t goes to 0.

Looking at various distribution one sees that when the distribution is exponential, W is constant; when the distribution is normal and thus decays doubly exponentially, W(t) behaves like $\log^{-1}(1/t)$. I am curious how the asymptotic decay of $W(t)$ translates to the tail behavior of the probability distribution. For the tracy Widom distribution when you ask what is the probability that $X \ge M+t \sigma$ you get an expression like $e^{-t^{3/2}}$ and for the probability that $X \le M-t \sigma$ you get something like $e^{-t^3}$. I wonder how these tails behavior can be described in terms of $W_X(t)$ and $W'_X(t)$ respectively, where $W'_X(t)$ deals with conditioning on $X \le s$ rather than on $X \ge s$. So let me form a simple question:

Question: If $W_X(t)$ behaves like $\log^a (1/t)$, for some real $a>0$, what does it say about the tail behavior of $X$. (More generally, how the decay of $W$ translates to the decay of the distribution.)

Start with a continuous probability distribution given by a density function f(x). Let X be a real random variable whose distribution is given by the probability distribution.

I would like to ask about the following specific way to describe the distribution in terms of the variance of tails.

Given a number t between 0 and 1 let s be such that the probability that $X\ge s$ equals t. (In other words, $\int_s^{\infty}f(x)=t$.) Let $W_X(t)$ be the variance of $X$ conditioned on $X\ge s$.

The function $W_X(t)$ seems an interesting way to describe the original probability distribution, and its tail behavior. I wonder if there is an explicit useful way to move from $W$ back to $f$; Can you give a dictionary between the tail behavior of $f$ and of that of $W$; And an explicit description of $W$ for some "famous" probability distributions. (E.g. the normal distribution, and the Tracy Widom distribution.) And of course if this kind of "transform" is considered in the literature.

Start with a continuous probability distribution given by a density function f(x). Let X be a real random variable whose distribution is given by the probability distribution.

I would like to ask about the following specific way to describe the distribution in terms of the variance of tails.

Given a number t between 0 and 1 let s be such that the probability that $X\ge s$ equals t. (In other words, $\int_s^{\infty}f(x)=t$.) Let $W_X(t)$ be the variance of $X$ conditioned on $X\ge s$.

The function $W_X(t)$ seems an interesting way to describe the original probability distribution, and its tail behavior. I wonder if there is an explicit useful way to move from $W$ back to $f$; Can you give a dictionary between the tail behavior of $f$ and of that of $W$; And an explicit description of $W$ for some "famous" probability distributions. (E.g. the normal distribution, and the Tracy Widom distribution.) And of course if this kind of "transform" is considered in the literature.

UPDATE: Many thanks to Robert and Brendan to their answers. Robert's asymptotic expansion seem amazing. I will be happy to understand the asymptotics as t goes to 0.

Looking at various distribution one sees that when the distribution is exponential, W is constant; when the distribution is normal and thus decays doubly exponentially, W(t) behaves like $\log^{-1}(1/t)$. I am curious how the asymptotic decay of $W(t)$ translates to the tail behavior of the probability distribution. For the tracy Widom distribution when you ask what is the probability that $X \ge M+t \sigma$ you get an expression like $e^{-t^{3/2}}$ and for the probability that $X \le M-t \sigma$ you get something like $e^{-t^3}$. I wonder how these tails behavior can be described in terms of $W_X(t)$ and $W'_X(t)$ respectively, where $W'_X(t)$ deals with conditioning on $X \le s$ rather than on $X \ge s$. So let me form a simple question:

Question: If $W_X(t)$ behaves like $\log^a (1/t)$, for some real $a>0$, what does it say about the tail behavior of $X$. (More generally, how the decay of $W$ translates to the decay of the distribution.)

Source Link

asked Jan 16, 2012 at 19:14

24.7k
38
235
327

A Variance-Tail Description for Continuous Probability Distributions

Start with a continuous probability distribution given by a density function f(x). Let X be a real random variable whose distribution is given by the probability distribution.

I would like to ask about the following specific way to describe the distribution in terms of the variance of tails.

Given a number t between 0 and 1 let s be such that the probability that $X\ge s$ equals t. (In other words, $\int_s^{\infty}f(x)=t$.) Let $W_X(t)$ be the variance of $X$ conditioned on $X\ge s$.

The function $W_X(t)$ seems an interesting way to describe the original probability distribution, and its tail behavior. I wonder if there is an explicit useful way to move from $W$ back to $f$; Can you give a dictionary between the tail behavior of $f$ and of that of $W$; And an explicit description of $W$ for some "famous" probability distributions. (E.g. the normal distribution, and the Tracy Widom distribution.) And of course if this kind of "transform" is considered in the literature.

pr.probability co.combinatorics