User shai covo - MathOverflow

A simple decomposition for fractional Brownian motion with parameter $H<1/2$

2010-11-02T08:11:21Z

Background

Let $X = \{X(t):t \geq 0\}$ be a (standard, real-valued) fractional Brownian motion (fBm) with parameter $H \in (0,1)$ , i.e., a continuous centered Gaussian process with covariance function given, for $0 \leq s \leq t$ , by $$C_X {(s,t)} := {\rm E}[X(s)X(t)] = \frac{1}{2}[t^{2H} + s^{2H} - (t - s)^{2H} ].$$ Writing $C_X {(s,t)}$ as $C_X {(s,t)} = \frac{1}{2}[t^{2H} - (t - s)^{2H} ] + \frac{1}{2}s^{2H}$ , gives rise to the decomposition of $X$ as $X = Y + Z$ , where $Y$ is a centered Gaussian process with covariance function $C_Y {(s,t)} = \frac{1}{2} [t^{2H} - (t - s)^{2H}]$ , independent of a time-changed Brownian motion $Z$ (specifically, $Z(t)=W(t^{2H}/2)$ , where $W$ is a standard BM). However, in order for $C_Y$ to be a valid covariance function it must be nonnegative definite. As indicated by numerical results (and can probably be easily proved), this is not the case for $H>1/2$ . For $H<1/2$ , on the other hand, $C_Y$ is the covariance function of some interesting Gaussian process arising in the setting of Gaussian random fields. Since I plan to write a paper on this apparently new subject, I find it sensible not to give too much details here (maybe I'll add some details later on).

Now to my questions. Have you encountered the aforementioned decomposition in the literature? (I haven't.) Does it correspond to some known (e.g., integral) representation of fBm? Can you think of some application of it? Finally, can you find a simple/useful representation for the process $Y$ in that decomposition (simple/useful compared to the fBm case)?

Answer by Shai Covo for Integral over error function and normal distribution

2011-04-06T14:11:52Z

This is too long to be a comment.

Let $X$ and $Y$ be independent ${\rm N}(\mu,\sigma_2)$ and ${\rm N}(0,q^2)$ rv's, respectively. Since $X+Y \sim {\rm N}(\mu,q^2+\sigma^2)$, it is equal in distribution to $Z + \mu$, where $Z \sim {\rm N}(0,q^2+\sigma^2)$. Hence,
$$ {\rm P}(X + Y \le \theta ) = {\rm P}(Z \le \theta - \mu ) = \frac{1}{{\sqrt {2\pi (\sigma ^2 + q^2 )} }}\int_{ - \infty }^{\theta - \mu } {e^{ - z^2 /[2(\sigma ^2 + q^2 )]} \,{\rm d}z} . $$ On the other hand, by the law of total probability (conditioning on $X$), we have $$ {\rm P}(X + Y \le \theta ) = \int_{ - \infty }^\infty {{\rm P}(Y \le \theta - x)\frac{1}{{\sqrt {2\pi \sigma ^2 } }}e^{ - (x - \mu )^2 /(2\sigma ^2 )} \,{\rm d}x}. $$ Therefore, $$ {\rm P}(X + Y \le \theta ) = \int_{ - \infty }^\infty {\bigg[\int_{ - \infty }^{\theta - x} {\frac{1}{{\sqrt {2\pi q^2 } }}e^{ - y^2 /(2q^2 )} \,{\rm d}y\bigg]\frac{1}{{\sqrt {2\pi \sigma ^2 } }}e^{ - (x - \mu )^2 /(2\sigma ^2 )} \,{\rm d}x} }.
$$ So, $$ \int_{ - \infty }^\infty {\bigg[\int_{ - \infty }^{\theta - x} {\frac{1}{{\sqrt {2\pi q^2 } }}e^{ - y^2 /(2q^2 )} \,{\rm d}y\bigg]\frac{1}{{\sqrt {2\pi \sigma ^2 } }}e^{ - (x - \mu )^2 /(2\sigma ^2 )} \,{\rm d}x} }, $$ which may correspond to the left-hand side expression in the question, is equal to $$ \frac{1}{{\sqrt {2\pi (\sigma ^2 + q^2 )} }}\int_{ - \infty }^{\theta - \mu } {e^{ - z^2 /[2(\sigma ^2 + q^2 )]} \,{\rm d}z}, $$ which may correspond to the right-hand side expression in the question (where $s^2$ should be $\sigma^2$).

Answer by Shai Covo for Weierstrass' function and Brownian motion

2011-03-21T19:21:25Z

You might find this account useful. In particular, see the end of Section 1, page 7, and the end of Section 4.

Answer by Shai Covo for Markov random field with continuous index set

2011-03-13T23:35:07Z

In general, the simplest extension of (real-valued) one-parameter processes is to (real-valued) multi-parameter processes, where the index set is $\mathbb{R}^n_+$ (usually $n=2$). This includes, for example, the $n$-parameter Brownian sheet, and more generally, $n$-parameter L\'evy sheets (or processes). However, such processes are much simpler than general set-indexed processes.

You'll probably find the following paper very useful (especially the introduction, since the set-indexed framework might be too heavy): A Markov Property For Set-Indexed Processes.

Answer by Shai Covo for best approximation to the LambertW(x) or exp(LambertW(x))

2011-03-08T17:19:22Z

Extended answer

The approximation described below is original, explicit (in some sense), and very accurate. It is closely related to this question (second paragraph).

So, you want to approximate the solution $W(x)$ of $x=W(x)e^{W(x)}$, for large $x$ (the order of $x$ does not play a very significant role in what follows). First, define $$ \varphi (x,r) = 1 + \sum\limits_{k = 1}^{\left\lceil r \right\rceil } {\frac{{x^k [r - (k - 1)]^k }}{{k!}}} . $$ Now, consider the following series of approximations, where $r$ is assumed sufficiently large. The first one is $$ \tilde W^1 (x,r) = \frac{1}{r}\ln \varphi (x,r). $$ Subsequent approximations are defined recursively by $$ \tilde W^{n + 1} (x,r) = \frac{1}{r}\ln \bigg[\frac{{\tilde W^n (1 + \tilde W^n )}}{x}\varphi (x,r)\bigg]. $$

Example. For $x=2000$, even $r$ as low as $80$ gives quite accurate results: $$ \tilde W^5 (2000,80) \approx 5.83673149492073 $$ and $$ \tilde W^6 (2000,80) \approx 5.836731494908671, $$ while the exact solution (according to Wims Function Calculator) is $$ W(2000) = 5.836731494908178747.... $$ Based on many numerical results, this approximation seems quite interesting. Here is one further example. The Omega constant $\Omega$ is the value of $W(1)$: $$ \Omega = W(1) \approx 0.5671432904097838729999686622. $$ With $r$ as low as $30$, we already get the following impressive approximations: $$ \tilde W^1 (1,30) \approx 0.5710729200334063, $$ $$ \tilde W^2 (1,30) \approx 0.5674569334624368, $$ $$ \tilde W^3 (1,30) \approx 0.5671683899602143, $$ $$ \tilde W^4 (1,30) \approx 0.5671452994467842, $$ $$ \tilde W^5 (1,30) \approx 0.5671434512213455, $$ $$ \tilde W^6 (1,30) \approx 0.5671433032818183, $$ $$ \tilde W^7 (1,30) \approx 0.5671432914401158, $$ $$ \tilde W^8 (1,30) \approx 0.567143290492256, $$ $$ \tilde W^9 (1,30) \approx 0.5671432904163853, $$ $$ \tilde W^{10} (1,30) \approx 0.5671432904103123, $$ $$ \tilde W^{11} (1,30) \approx 0.5671432904098261, $$ $$ \tilde W^{12} (1,30) \approx 0.5671432904097873. $$ So, $\tilde W^{12} (1,30) - W(1) \approx 3 \times 10^{-15}$. It is interesting to compare this sophisticated approximation with the standard one obtained from the converging sequence $\Omega_n \to \Omega$ defined by $\Omega_{n+1} = e^{-\Omega_n}$ (with initial value $\Omega_0$). For example, with $\Omega_0 = 0.5$, we only get $$ \Omega_1 \approx 0.6065306597126334, $$ $$ \Omega_2 \approx 0.545239211892605, $$ $$ \Omega_3 \approx 0.5797030948780683, $$ $$ \Omega_4 \approx 0.5600646279389019, $$ $$ \Omega_5 \approx 0.5711721489772151, $$ $$ \Omega_6 \approx 0.5648629469803235, $$ $$ \Omega_7 \approx 0.5684380475700662, $$ $$ \Omega_8 \approx 0.5664094527469208, $$ $$ \Omega_9 \approx 0.5675596342622424, $$ $$ \Omega_{10} \approx 0.5669072129354714, $$ $$ \Omega_{11} \approx 0.5672771959707785, $$ $$ \Omega_{12} \approx 0.5670673518537281. $$

Answer by Shai Covo for Is there any result discribing the value of the correlation of a measurable function of `$X$` and itself: `$corr(f(X),X)$` ?

2011-02-24T08:10:45Z

At least there is a simple result for the case $f(x)=e^{tx}$. Under suitable conditions, we have $$ {\rm corr}(X,e^{tX} ) = \frac{{{\rm E}[Xe^{tX} ] - \mu _X {\rm E}[e^{tX} ]}}{{\sigma _X \sqrt {{\rm E}[e^{2tX} ] - {\rm E}^2 [e^{tX} ]} }} = \frac{{m'_X (t) - \mu _X m_X (t)}}{{\sigma _X \sqrt {m_X (2t) - m_X^2 (t)} }}, $$ where $m_X (\cdot)$, $\mu_X$, and $\sigma^2_X$ denote the moment-generating function, the expectation, and the variance of $X$, respectively.

For example, for $X$ exponential with mean $1/\lambda$ (hence $m_X (\tau) = \frac{\lambda }{{\lambda - \tau}}$, $\tau \lt \lambda$, $\sigma_X = 1/\lambda$), this yields $$ {\rm corr}(X,e^{tX} ) = \frac{{\lambda t}}{{(\lambda - t)^2 \sqrt {\frac{\lambda }{{\lambda - 2t}} - (\frac{\lambda }{{\lambda - t}})^2 } }} , \;\; t \in ( - \infty ,\lambda /2)\backslash \lbrace 0 \rbrace . $$ (Using ${\rm E}[X^n] = n!/\lambda^n$, we can also find that ${\rm corr}(X,X^n) = \frac{n}{{\sqrt {{2n \choose n} - 1} }}$.) Note that $\lim _{t \to 0 \pm } {\rm corr}(X,e^{tX} ) = \pm 1$ and $\lim _{t \uparrow \lambda /2} {\rm corr}(X,e^{tX} ) = 0$. (For the former, consider $e^{tX} \approx 1 + tX$.)

The general formula for ${\rm corr}(X,e^{tX} )$ may help you reach some conclusion. In particular, note that the functions $f(x)=e^{tx}$ and $m_X (t)$ are often closely related, see here, for example.

Answer by Shai Covo for Summation of an expression

2011-02-14T22:43:17Z

REVISED ANSWER.

In retrospect, deriving the approximation is quite easy. Indeed, $$ \sum\limits_{j = 1}^n {\frac{{j^k }}{{(j - 1)!}}} = e\sum\limits_{j = 1}^n {e^{ - 1} \frac{{j^{k + 1} }}{{j!}}} \approx e\sum\limits_{j = 0}^\infty {e^{ - 1} \frac{{j^{k + 1} }}{{j!}}} = eB_{k + 1}. $$ (For large $k$, you may consider Asymptotic limit and bounds.)

ORIGINAL ANSWER.

Assume that $n$ is sufficiently large. Since $$ \sum\limits_{j = 1}^n {\frac{{j^k }}{{(j - 1)!}}} = \sum\limits_{j = 0}^{n - 1} {\frac{{(j + 1)^k }}{{j!}}} , $$ the problem reduces to approximating $\sum\nolimits_{j = 0}^{n - 1} {\frac{{j^m }}{{j!}}} $, for $0 \leq m \leq k$. Now, $e^{ - 1} \sum\nolimits_{j = 0}^\infty {\frac{{j^m }}{{j!}}} $ is the $m$-th moment of the Poisson distribution with mean $1$. For the latter, see, for example, this.

EDIT: Specifically, the approximation (with the right-hand side being an upper bound) is $$ \sum\limits_{j = 1}^n {\frac{{j^k }}{{(j - 1)!}}} \approx e\sum\limits_{m = 0}^k {{k \choose m}B_m }, $$ where $B_m$ is the $m$-th Bell number (a list is given here). Numerical results indicate that this approximation is very accurate, even for moderate values of $n$. For example, the absolute error for $n=15$, $k=3$ is $\approx 3.4 \times 10^{-9}$; for $n=20$, $k=5$ is $\approx 1.8 \times 10^{-12}$; for $n=20$, $k=7$ is $\approx 7.9 \times 10^{-10}$; for $n=23$, $k=9$ is $\approx 5.8 \times 10^{-11}$.

EDIT: If only the relative error is concerned, then the approximation is quite accurate even for relatively small $n$ values (of course, depending on $k$). For example, for $n=8$, $k=4$: $\approx 141.156$ compared to $\approx 141.351$; for $n=9$, $k=5$: $\approx 551.484$ compared to $\approx 551.811$; for $n=10$, $k=6$: $\approx 2383.359$ compared to $\approx 2383.933$.

EDIT: As Mike Spivey observed, using the identity $$ \sum\limits_{m = 0}^k {{k \choose m}B_m } = B_{k + 1}, $$ the above approximation can be simplified greatly to $$ \sum\limits_{j = 1}^n {\frac{{j^k }}{{(j - 1)!}}} \approx e B_{k+1}. $$

Answer by Shai Covo for Fonction spéciale

2011-02-11T10:19:00Z

Original proof:

Let's consider the second integral with $\alpha = 0$. Noting, on the one hand, that the integrand is not real for $t \gt 1$ if, for example, $\gamma=2.5$, and, on the other hand, observing its relation to the density function of the ${\rm Beta}(\beta,\gamma-1)$ distribution, it seems likely that the integral (with $\alpha = 0$) is actually $$ I(s;\beta,\gamma) = \int_0^1 {e^{ - st} t^{\beta - 1} (1 - t)^{\gamma - 2} \,{\rm d}t}, $$ where $\beta \gt 0$ and $\gamma \gt 1$. Now, $\frac{{\Gamma (\beta + \gamma - 1)}}{{\Gamma (\beta )\Gamma (\gamma - 1)}} I(s;\beta,\gamma)$, for $s \geq 0$, is the Laplace transform of the ${\rm Beta}(\beta,\gamma-1)$ distribution. The moment-generating function is given by $$ m(s) = 1 + \sum\limits_{k = 1}^\infty {\bigg(\prod\limits_{r = 0}^{k - 1} {\frac{{\beta + r}}{{\beta + \gamma - 1 + r}}} \bigg)\frac{{s^k }}{{k!}}}. $$ Writing $$ \frac{{\Gamma (\beta + \gamma - 1)}}{{\Gamma (\beta )\Gamma (\gamma - 1)}} I(s;\beta,\gamma) = m(-s) $$ gives $$ I(s;\beta,\gamma) = \frac{{\Gamma (\beta )\Gamma (\gamma - 1)}}{{\Gamma (\beta + \gamma - 1)}}\bigg[1 + \sum\limits_{k = 1}^\infty {\bigg(\prod\limits_{r = 0}^{k - 1} {\frac{{\beta + r}}{{\beta + \gamma - 1 + r}}} \bigg)\frac{{(-s)^k }}{{k!}}}\bigg]. $$

It turns out that this result is well known (cf. Eq. (3) here):

Using the standard notation in the theory of special functions (in particular the hypergeometric function), $I(s;\beta,\gamma)$ can be written as $$ I(s;\beta,\gamma) = \frac{{\Gamma (\beta )\Gamma (\gamma - 1)}}{{\Gamma (\beta + \gamma - 1)}} \bigg[ 1 + \sum\limits_{k = 1}^\infty {\frac{{(\beta )_k }}{{(\beta + \gamma - 1)_k }}\frac{{( - s)^k }}{{k!}}} \bigg] $$ ($(x)_n$ represents the rising factorial). Finally, in terms of the confluent hypergeometric function of the first kind, $$ I(s;\beta,\gamma) = \frac{{\Gamma (\beta )\Gamma (\gamma - 1)}}{{\Gamma (\beta + \gamma - 1)}} {}_1F_1 (\beta ;\beta + \gamma - 1; - s). $$

Answer by Shai Covo for How I determine the probability that an unknown probability value is greater than others in a set?

2011-02-09T15:11:59Z

In view of the [algorithms] tag (and since you are a software engineer), perhaps you'll be satisfied with the following answer. Assume that $X_i$ are independent ${\rm Beta}(\alpha_i,\beta_i)$ variables. Then, you can evaluate the probability ${\rm P}(X_i = \max \lbrace X_1 , \ldots ,X_n \rbrace )$ using Monte Carlo simulations, as follows. Obviously, the problem amounts to simulating a ${\rm Beta}(\alpha,\beta)$ variable. This can be done simply as follows, according to Example 2.11 in the book Monte Carlo statistical methods (see references therein). If $U$ and $V$ are independent ${\rm uniform}[0,1]$ variables, then the distribution of $\frac{{U^{1/\alpha } }}{{U^{1/\alpha } + V^{1/\beta } }}$ conditional on $U^{1/\alpha } + V^{1/\beta } \le 1$ is the ${\rm Beta}(\alpha,\beta)$ distribution. As noted in that example, this result does not provide a good algorithm to generate ${\rm Beta}(\alpha,\beta)$ variables for large values of $\alpha$ and $\beta$ (because of the constraint on $U^{1/\alpha } + V^{1/\beta }$). But if your $\alpha_i$ and $\beta_i$ are not large, you might find this simple algorithm useful enough (depending on the accuracy you wish to achieve).

EDIT: This approach may be particularly useful for values $\alpha_i,\beta_i \in (0,1)$, for two reasons. First, this increases the probability that $U^{1/\alpha } + V^{1/\beta } \le 1$ (that is, the pair $(U,V)$ is not rejected). Second, the ${\rm Beta}(\alpha,\beta)$ density function is not bounded if $\alpha \in (0,1)$ or $\beta \in (0,1)$, and so a tractable analytical expression for ${\rm P}(X_i = \max \lbrace X_1 , \ldots ,X_n \rbrace )$ is not likely to be found in this case. Of course, everything changes if the parameters are integers...

Answer by Shai Covo for Probabilistic Proofs of Analytic Facts

2011-01-10T05:41:09Z

How about the Convolution Theorem, which can be seen as a consequence of $$ {\rm E}[e^{iu(X+Y)}]={\rm E}[e^{iuX}]{\rm E}[e^{iuY}],\;\; u \in \mathbb{R}, $$ where $X$ and $Y$ are independent random variables.

Answer by Shai Covo for what will be the distribution of ratio of correlated gamma distributed random variables?

2010-10-25T18:49:59Z

First of all, since $R=(X+C)/(X+Y)$ (and $X$ and $Y$ are independent gamma variables), then the valid range of $R$ is a priori $(0,\infty)$ . The density of $(R,S)$ is given by $f_{R,S} (r,s) = f_X (sr - C)f_Y (s - sr + C)s$ , where $f_X$ and $f_Y$ are the densities of $X$ and $Y$ (see the remark below). This leads to $sr-C > 0$ and $s-sr+C > 0$ . Thus, $s > C/r$ and $s(r-1) < C$ . Hence, if $r > 1$ , then $C/r < s < C/(r-1)$ , while if $0 < r < 1$ , then $C/r < s < \infty$ . So, for $0 < r < 1$ you would use $f_R (r) = \int_{C/r}^\infty {f_{R,S} (r,s)\,{\rm d}s}$ , and for $r > 1$, $f_R (r) = \int_{C/r}^{C/(r-1)} {f_{R,S} (r,s)\,{\rm d}s}$. In general, a nice expression for $f_R (r)$ is not likely to be found. [Minor point: in view of the title, note that $X+Y$ is, in general, not gamma distributed.]

Remark: As Didier observed, a factor of $s$ was missing in the original expression for $f_{R,S}(r,s)$ (now fixed). The density $f_{R,S}(r,s)$ is found as follows. $(X,Y)$ has density $f_{X,Y}(x,y)=f_X (x) f_Y (y)$ . Noting that $X=SR-C$ and $Y=S-SR+C$ , it follows by the standard formula for transformation of rv's that $f_{R,S} (r,s)$ is given by $f_{X,Y}(sr-C,s-sr+C)$ times $|J(r,s)|$ , where $J(r,s)$ is given by the determinant $\frac{{\partial (sr - C)}}{{\partial r}}\frac{{\partial (s - sr + C)}}{{\partial s}} - \frac{{\partial (sr - C)}}{{\partial s}}\frac{{\partial (s - sr + C)}}{{\partial r}} = s$ . Thus, $f_{R,S} (r,s)$ is $s$ times the original expression.

Answer by Shai Covo for Results on derivatives with respect to the parameter of Modified Bessel Function

2011-01-09T22:19:19Z

See, for example, here. As noted in this link, further details can be found in this paper.

Answer by Shai Covo for An application of Baire category theorem

2011-01-09T20:45:17Z

For a proof of the much stronger result indicated above, see Page 53 here; the theorem states the following:

Let $f(x)$ be $C^\infty$ on $(c,d)$ such that for every point $x$ in the interval there exists an integer $N_x$ for which $f^{(N_x)}(x)=0$; then $f(x)$ is a polynomial.

Answer by Shai Covo for An optimization problem

2011-01-06T21:31:19Z

Here's an interesting/natural observation which might be useful. Let $U$ be a uniform$(0,1/2)$ random variable, so that $U$ has density function $f(x)=2$, $0 \lt x \lt 1/2$, and define a function $Y$ by $Y(x)=(1/2-x)Q(x)$, $0 \lt x \lt 1/2$. Then, $M=1/12 + \frac{1}{2}{\rm E}(Y^2 ) - {\rm E}^2 (Y)$. (Interestingly, a uniform$(0,1)$ random variable has variance equal to $1/12$.)

Answer by Shai Covo for Differentiation of a series of increasing functions

2011-01-04T17:51:16Z

Yes, see Theorem 4.1 on p. 177 of this book.

Answer by Shai Covo for Process for a Gamma distribution with non integer shape parameter

2011-01-02T23:43:50Z

Here is some interesting, though natural, view of the problem (including a small but quite elegant result at the end). Denote by ${\rm Gamma}(k,\lambda)$ the gamma distribution with density function $f(x;k,\lambda) = \lambda ^k x^{k - 1} e^{ - \lambda x}/ \Gamma (k)$ , $x>0$ . Here $k$ and $\lambda$ are arbitrary positive constants. Clearly, the problem reduces to the case $0<k<1$ . Indeed, if $k$ is non-integer and greater than $1$ , then a ${\rm Gamma}(k,\lambda)$ random variable can be viewed as a sum of independent ${\rm Gamma}(\left\lfloor k \right\rfloor,\lambda)$ and ${\rm Gamma}(k - \left\lfloor k \right\rfloor,\lambda)$ random variables. Now, let's consider a gamma process $X = \lbrace X_t : t \geq 0\rbrace$ such that $X_1 \sim {\rm exponential}(\lambda)$ . That is, $X$ is a process with stationary independent increments, starting at $0$ and having right-continuous with left limits sample paths (i.e., a L\'evy process), such that $X_t - X_s \sim {\rm Gamma}(t-s,\lambda)$ , for any $0 \leq s < t$ . The process $X$ is a pure jump process, having positive jumps only, and it holds $X_t = \sum\nolimits_{0 < s \le t} {\Delta X_s } $ , where $\Delta X_s = X_s - X_{s-}$ . The number of jumps is a.s. infinite countable in any time interval, no matter how small (yet, $\Delta X_t =0$ a.s., for any fixed $t>0$ ). The process $X$ is characterized by its jump measure (L\'evy measure) given by $\nu({\rm d}x)=x^{-1}e^{-\lambda x}\,{\rm d}x$ : given a time interval $[t_1,t_2]$ , the number of jumps of $X$ whose sizes lie in $[a,b] \subset (0,\infty)$ is Poisson distributed with mean $(t_2-t_1)\int_a^b {x^{ - 1} e^{ - \lambda x } \,{\rm d}x}$ (the key concept here is Poisson random measure). Hence, roughly speaking, the process usually increases by tiny jumps (whose sum is negligible in practice).

Now, consider the process $X$ on the time interval $[0,1]$ . On the one hand, $X_1$ is the (infinite) sum of jumps of $X$ up to time $t=1$ . On the other hand, $X_1$ is distributed as the waiting time until the first occurrence in a Poisson process with rate $\lambda$ . This is already quite interesting. Returning to the original problem, we can view a ${\rm Gamma}(k,\lambda)$ random variable, $0<k<1$ fixed, as the random variable $X_k$ , i.e., the sum of jumps of $X$ up to time $k$ (note that $\lim _{k \uparrow 1} X_k = X_1 $ , a.s.). So, in some respect, this already gives a process-based description of the meaning of a Gamma distribution for a non-integer parameter. But much more can be said, using the fact that if $X$ and $Y$ are independent ${\rm Gamma}(\alpha,\lambda)$ and ${\rm Gamma}(\beta,\lambda)$ rv's, respectively, then the ratio $X/(X+Y)$ has a ${\rm Beta}(\alpha,\beta)$ distribution. Namely, the ratio $R = X_k / X_1$ , $0<k<1$ , is distributed as a ${\rm Beta}(k,1-k)$ random variable, implying that $R$ has density function $f_{R} (x;k) = x^{k-1}(1-x)^{-k}\sin(\pi k)/\pi$ , $0<x<1$ , where we have used $\Gamma(k)\Gamma(1-k) = \pi/\sin(\pi k)$ . In particular, $f_R (x;1/2) = 1/(\pi \sqrt {x(1 - x)} )$ , $0<x<1$ .

Answer by Shai Covo for Order statistics: probability random variable is k-th out of n when ordered.

2010-12-30T23:22:34Z

Suppose that $X_1$ is supported on $[a,b]$ with density function $f$ , and that $X_2,\ldots,X_n$ are i.i.d., independent also of $X_1$ , with distribution function $G$ . Let $P_k$ denote the probability that $X_1$ is the $k$ th largest among the $n$ rv's. Then, with a somewhat loose notation, $P_k = {n-1 \choose k-1}{\rm P}([X_1>X_2,\ldots,X_{n-k+1}],[X_1<X_{n-k+2},\ldots,X_n]).$ (The $k-1$ comes from $n-(n-k+2)+1$ .) By the law of total probability (conditioning on $X_1$ ), we get, using the independence assumptions, $ P_k = {n-1 \choose k-1} \int _a^b {[G(s)]^{n - k} [1 - G(s)]^{k - 1}f(s)\,{\rm d}s}.$

Remark: This answer was composed completely independently of the one given at stats.stackexchange.

Answer by Shai Covo for $E(X_1 | X_1 + X_2)$, where $X_i$ are (integrable) independent infinitely divisible rv's "of the same type"

2010-12-28T11:05:43Z

The following was motivated by Didier's comment given below my first answer.

On the one hand, the role of infinite divisibility (ID) might not seem important in our context, in view of the following general example (and, moreover, part of the next paragraph). If $Z$ is any integrable random variable, and if $a/(a+b)$ is rational, say $a/(a+b)=n_1/(n_1+n_2)$ with $n_1,n_2 \in \mathbb{N}$, then letting $X = \sum\nolimits_{i = 1}^{n_1 } {Z_i }$ and $Y = \sum\nolimits_{i = n_1+1}^{n_1+n_2 } {Z_i }$, where $Z_i$ are independent copies of $Z$, we have ${\rm E}( X|X + Y)=\frac{a}{{a + b}}(X + Y)$. As a side note, it is worth noting here that for $X \sim {\rm binomial}(n_1,p)$, $Y \sim {\rm binomial}(n_2,p)$ this gives ${\rm E}(X|X+Y)=\frac{{n_1 }}{{n_1 + n_2 }}(X+Y)$, a result which might be quite difficult to obtain directly, that is by calculating $\sum\nolimits_{k = 0}^{n_1 } {k{\rm P}(X = k|X + Y = n)} $ (this can be a challenging exercise for students).

On the other hand, consider the following question. Suppose that $X$ and $Y$ are independent integrable random variables with characteristic functions $\varphi_X$ and $\varphi_Y$, respectively, and $a$ and $b$ are positive real constants. Is it true that ${\rm E}(X|X+Y) = \frac{a}{{a + b}}(X + Y)$ if and only if $\varphi_Y = \varphi_X^{b/a}$? If $X$ is ID, then the condition $\varphi_Y = \varphi_X^{b/a}$ implies ${\rm E}(X|X+Y) = \frac{a}{{a + b}}(X + Y)$. Didier's answer suggests that this is true in general, and moreover that the opposite implication might be true as well, since it gives rise to the differential equation $b\frac{{\varphi'_X }}{{\varphi _X }} = a\frac{{\varphi' _Y }}{{\varphi _Y }}$, hence to $\varphi_Y = \varphi_X^{b/a}$ (note that $\varphi _X (0) = \varphi _Y (0) = 1$). It might be important to point out here that the characteristic function of an ID random variable has no zero. However, if $X$ is not ID, then $\varphi_X^{b/a}$ might not be a characteristic function. Indeed, if $\varphi {}_X^c$ is a characteristic function for all $c>0$, then from $\varphi _X = (\varphi _X^{1/n} )^n$ $\forall n$ it would follow that $X$ is ID. So, it seems that infinite divisibility does play an important role in our context.

Finally, do you think that indeed ${\rm E}(X|X+Y) = \frac{a}{{a + b}}(X + Y)$ if and only if $\varphi_Y = \varphi_X^{b/a}$? It is quite an important result, if it is true...

$E(X_1 | X_1 + X_2)$, where $X_i$ are (integrable) independent infinitely divisible rv's "of the same type"

2010-11-24T00:24:23Z

The following is inspired by this recent question on math.stackexchange. Two standard exercises in conditional expectation are to find ${\rm E}(X_1|X_1+X_2)$ where: 1) $X_i$ , $i=1,2$ , are independent ${\rm N}(0,\sigma_i^2)$ rv's; 2) $X_i$ , $i=1,2$ , are independent Poisson( $\lambda_i$ ) rv's. The solutions are given by $\frac{{\sigma _1^2 }}{{\sigma _1^2 + \sigma _2^2 }}(X_1 + X_2)$ and $\frac{{\lambda _1 }}{{\lambda _1 + \lambda _2 }}(X_1 + X_2)$ , respectively. A proof for case 1) is given on math.stackexchange. For case 2) we have ${\rm E}(X_1|X_1 + X_2 = n) = \sum\limits_{k = 0}^n {k{\rm P}(X_1 = k|X_1 + X_2 = n)}.$ A straightforward calculation shows that the right-hand side sum is equal to $\sum\limits_{k = 0}^n {k{n \choose k}\bigg(\frac{{\lambda _1 }}{{\lambda _1 + \lambda _2 }}\bigg)^k \bigg(\frac{{\lambda _2 }}{{\lambda _1 + \lambda _2 }}\bigg)^{n - k} },$ which is the expectation of the binomial distribution with parameters $n$ and $\lambda_1 / (\lambda_1 + \lambda_2)$ , hence given by $ n \lambda_1 / (\lambda_1 + \lambda_2)$ . The result for case 2) is thus proved. In this context, what is common to the normal and Poisson distributions is that both are infinitely divisible (ID). More specifically, the characteristic function of $X_i \sim {\rm N}(0,\sigma_i^2)$ is given by ${\rm E}[{\rm e}^{{\rm i}zX_i} ] = {\rm e}^{\sigma _i^2 ( - z^2 /2)}$ , and that of $X_i \sim {\rm Poisson}(\lambda_i)$ by ${\rm E}[{\rm e}^{{\rm i}zX_i} ] = {\rm e}^{\lambda _i ({\rm e}^{{\rm i}z} - 1)}$ . Now, consider integrable, ID, independent rv's $X_i$, $i=1,2$, with characteristic functions of the form ${\rm E}[{\rm e}^{{\rm i}zX_i} ] = {\rm e}^{c_i \psi(z)}$ , $c_i > 0$ (loosely speaking, the characteristic function of an arbitrary ID rv is of that form). In view of the normal and Poisson examples considered above (the former requires somewhat tedious algebra for the solution), and the fact that many important rv's fall into the general category of integrable ID rv's (e.g., gamma rv's), it would be very useful to have the following result: ${\rm E}(X_1 | X_1 + X_2) = \frac{{c _1 }}{{c_1 + c_2 }}(X_1 + X_2)$ . In fact, I have proved it recently. Now to my questions: 1) Have you encountered this result before? 2) Can you provide a rigorous but simple proof of it? 3) Can you provide some intuition?

EDIT: 1) Here's another interesting example: if $X_i \sim {\rm Gamma}(c_i,\lambda)$ , $i=1,2$ , so that $X_i$ has density $f_{X_i } (x) = \lambda ^{c_i } {\rm e}^{ - \lambda x} x^{c_i - 1} /\Gamma (c_i )$ , $x > 0$ , then ${\rm E}(X_1 | X_1 + X_2) = \frac{{c_1 }}{{c_1 + c_2 }}(X_1 + X_2 )$ . 2) It is very instructive to reformulate the result in terms of L\'evy processes: if $X = \{ X(t): t \geq 0 \}$ is an integrable L\'evy process, then ${\rm E}[X(s)|X(t)] = \frac{s}{t}X(t)$ , $0 < s < t$ .

EDIT: The "direct" solution for the gamma case considered above is now given here. This shows, once more, the effectiveness of the general formula.

EDIT: A complete solution is given in my first (according to date) answer below.

EDIT: An important extension is considered in my second answer below.

Answer by Shai Covo for $E(X_1 | X_1 + X_2)$, where $X_i$ are (integrable) independent infinitely divisible rv's "of the same type"

2010-12-06T23:44:40Z

First of all, considering the responses from this site, it seems that this result is not well-known (even among specialists), though very useful and relatively easy to derive. So, it was worth posting this here, and it is worth considering this a little further.

I'll begin with Didier's answer, which corresponds to the characteristic functions formulation (original question). The main point, using Didier's notation, is that $(a+b){\rm E}(X|S) = a S$ (what we want to show) is implied by $(a + b){\rm E}(X{\rm e}^{{\rm i}tS} ) = a{\rm E}(S{\rm e}^{{\rm i}tS} )$ for every $t \in \mathbb{R}$ . Indeed, the latter condition implies $(a + b){\rm E}(X \mathbf{1}_A ) = a{\rm E}(S \mathbf{1}_A )$ for any $A \in \sigma(S)$ , and thus, from the definition of conditional expectation, $(a+b){\rm E}(X|S) = a S$ . Now, as Didier described, showing that $(a + b){\rm E}(X{\rm e}^{{\rm i}tS} ) = a{\rm E}(S{\rm e}^{{\rm i}tS} )$ is very easy, under the assumption ${\rm E}({\rm e}^{{\rm i}tX}) = {\rm e}^{a \psi(t)}$ and ${\rm E}({\rm e}^{{\rm i}tY}) = {\rm e}^{b \psi(t)}$ . For completeness, the following point(s) should be noted here. $\frac{{\rm d}}{{{\rm d}t}}{\rm E}({\rm e}^{{\rm i}tX} ) = {\rm i}{\rm E}(X{\rm e}^{{\rm i}tX} )$ by virtue of the dominated convergence theorem (since $X$ is integrable; the same goes with respect to $Y$ ). So, ${\rm e}^{a \psi(t)}$ is differentiable, and from the fact that $\psi$ is continuous it follows that $\frac{{\rm d}}{{{\rm d}t}} {\rm e}^{a \psi(t)} = {\rm e}^{a \psi(t)} a \psi'(t)$ , which we needed for the proof. [Interestingly, this shows that if $X$ is an integrable ID rv, then the corresponding characteristic exponent, $\psi$ , is differentiable.] So overall, it seems that Didier indeed provided a rigorous but (relatively) simple proof.

Ori's answer, on the other hand, corresponds to the L\'evy process formulation. My original proof of the result completes Ori's answer (the beginning is essentially the same). Here it is. Suppose that $X$ is an integrable L\'evy process, and fix $0 < s < t$ . Assume first that $s/t=m/n$ , with $m,n \in \mathbb{N}$ . From $\sum\nolimits_{i = 1}^n {{\rm E}[X_{it/n} - X_{(i - 1)t/n} |X_t ]} = X_t$ we deduce that ${\rm E}[X_{t/n}|X_t]=X_t / n$ , and, in turn, ${\rm E}[X_s |X_t ] = (m/n)X_t = (s/t)X_t$ . If $s/t$ is irrational, let $(s_j)$ be a sequence such that $s_j \uparrow s$ with $s_j/t$ being rational. By an elementary property of L\'evy processes, $X_{s_j } \stackrel{{\rm a.s.}}{\rightarrow} X_s $. Define $X_s^* = \sup _{u \in [0,s]} |X_u |$; thus $|X_{s_j}|\leq X_s^*$ $\forall j$. Since, by assumption, ${\rm E}[|X_s|]<\infty$ , we conclude from Theorem 25.18 in the classical book "L\'evy Processes and Infinitely Divisible Distributions" (by Sato) that also ${\rm E}[X_s^*]<\infty$ . Hence, by the dominated convergence theorem for conditional expectations, ${\rm E}[X_{s_j } |X_t ] \stackrel{{\rm a.s.}}{\rightarrow} {\rm E}[X_s |X_t ]$ . Since $s_j/t$ is rational, ${\rm E}[X_{s_j } |X_t ]=(s_j/t)X_t$ . Thus, ${\rm E}[X_s |X_t ] = (s/t)X_t$ .

Finally, Louigi's approach may be useful in a more general setting. In this context, I find it interesting to consider ${\rm E}(X_s | X_t)$ ( $0 < s < t$ ) for general processes (cf. its counterpart ${\rm E}(X_t | X_s)$ ). Any ideas?

Answer by Shai Covo for How to reading of an integral? Bernoulli trials with variable success rate, p

2010-12-26T14:02:51Z

The result corresponds to a special case of the Beta-binomial distribution, which can be generalized as follows.

Suppose that $Y \sim {\rm Beta}(\alpha,\beta)$, $\alpha,\beta>0$ real; thus $Y$ has density $f_Y{(p)} = p^{\alpha - 1} (1 - p)^{\beta - 1} /{\rm B}(\alpha ,\beta )$, $p \in [0,1]$, where ${\rm B}(x,y) = \Gamma(x)\Gamma(y)/\Gamma(x+y)$ is the Beta function. Further suppose that $X \sim {\rm binomial}(N,Y)$, $N \in \mathbb{N}$ fixed, meaning that given $Y=p$, $X \sim {\rm binomial}(N,p)$. Then, $X$ has a Beta-binomial distribution with parameters $N$, $\alpha$, and $\beta$. By the law of total probability, the probability mass function of $X$ is given, for $k=0,1,\ldots,N$, by $$ {\rm P}(X=k) = {\rm E}[{\rm P}(X=k|Y)] = \int_0^1 {{N \choose k}p^k (1 - p)^{N - k} f_Y (p)\,{\rm d}p}. $$ Hence, $$ \int_0^1 {p^{k + \alpha - 1} (1 - p)^{N - k + \beta - 1} \,{\rm d}p} = \frac{{{\rm B}(\alpha ,\beta )}}{{{N \choose k}}}{\rm P}(X = k). $$ Explicitly, the left-hand side is given by $$ \int_0^1 {p^{k + \alpha - 1} (1 - p)^{N - k + \beta - 1} \,{\rm d}p} = {\rm B}(k + \alpha ,N - k + \beta ) = \frac{{\Gamma (k + \alpha )\Gamma (N - k + \beta )}}{{\Gamma (N + \alpha + \beta )}} $$ (say, by definition of the Beta function), but this is, of course, not the point here: the point is the relation to the Beta-binomial distribution. In the special case where $\alpha=\beta=1$, we have $Y \sim {\rm uniform}[0,1]$, and by substitution we find that ${\rm P}(X=k)=1/(N+1)$. Hence the "uniform$[0,1]$-binomial distribution" is simply the discrete uniform distribution on $\lbrace 0,1,\ldots,N \rbrace$.

Answer by Shai Covo for A type of stochastic jump process

2010-12-22T15:20:59Z

Let $\tau = \min \{ n \geq 1:X_1 + \cdots + X_n > K \}$ . Then $\tau$ is an integer-valued random variable, bounded from above by $K+1$ (since $X_i \geq 1$ ). Note that $\tau = n$ if and only if $\sum\nolimits_{i = 1}^{n - 1} {X_i } \le K$ and $\sum\nolimits_{i = 1}^{n} {X_i } > K$ . Thus, the event $\lbrace \tau = n \rbrace$ depends only on the values $X_1,\ldots,X_n$ . So, by definition, $\tau$ is a stopping time with respect to the sequence $X_1,X_2,\ldots$ . Now, $X_1,X_2,\ldots$ are i.i.d. with finite expectation $\mu$ , and $\tau$ is a stopping time for them. Moreover, ${\rm E}(\tau) < \infty$ since $\tau \leq K+1$ . Hence, by Wald's identity, ${\rm E}\bigg(\sum\limits_{i = 1}^\tau {X_i } \bigg) = {\rm E}(\tau )\mu \leq (K+1)\mu. $ So if we put $Y_\tau = \sum\nolimits_{i = 1}^\tau {X_i }$ , we get ${\rm E}(Y_\tau - K) = {\rm E}(Y_\tau) - K \leq (K+1)\mu - K.$

EDIT: Since $\tau \geq 1$ , we have $\mu - K \leq {\rm E}(Y_\tau - K) \leq (K+1)\mu - K. $

As we have seen above, the problem reduces to calculating ${\rm E}(\tau)$ . Put $S_n = \sum\nolimits_{i = 1}^n {X_i }$ ( $S_0 = 0$ ). Note that $ {\rm P}(\tau = n) = {\rm P}(S_{n - 1} \le K,S_n > K) = {\rm P}(S_{n - 1} \le K) - {\rm P}(S_n \le K). $ Hence, $ {\rm E}(\tau) = \sum\limits_{n = 1}^{K + 1} {n{\rm P}(\tau = n)} = \sum\limits_{n = 1}^{K + 1} n[{\rm P}(S_{n - 1} \le K) - {\rm P}(S_n \le K)] = \sum\limits_{n = 0}^K {{\rm P}(S_n \le K)}. $ So, we can write $ {\rm E}(\tau) = 1 + \sum\limits_{n = 1}^\infty {{\rm P}(S_n \le K)} = 1 + \sum\limits_{n = 1}^\infty {F^{(n)}(K)}, $ where $F^{(n)}$ is the distribution function of $S_n$ . For $t>0$ real, define $m(t) = \sum\nolimits_{n = 1}^\infty {F^{(n)}(t)}$ . From the theory of renewal processes, we know that $m(t) = {\rm E}(N_t)$ , where $\lbrace N_t:t \geq 0 \rbrace$ is a renewal process with inter-arrival times distributed according to the distribution of $X$ . $m(t)$ is called the renewal function. It may be worth noting that by the Elementary Renewal Theorem, $\mathop {\lim }\limits_{t \to \infty } \frac{{m(t)}}{t} = \frac{1}{\mu }. $ Returning to our original setting, we have $ {\rm E}(Y_\tau ) = {\rm E}(\tau )\mu = (1 + m(K))\mu. $ So, the problem reduces to calculating $m(K)$ .

EDIT: Elaborating on the relation to the framework of renewal theory.

For completeness and for general purposes, let us consider the problem in the (more general) setting of renewal theory. For this purpose, we replace $Y$ by $S$ , in accordance with the common notation used in renewal theory. Henceforth we suppose that $X_i$ are i.i.d. non-negative rv's with mean $\mu > 0$ , and set $S_n = \sum\nolimits_{i = 1}^n {X_i }$ . For $t \geq 0$ real, we set $\tau_t = \inf \{ n : S_n > t \}$ (thus further generalizing the case considered in the question). We now introduce the stochastic process $N = \lbrace N_t : t \geq 0 \rbrace$ , defined by $N_t = \sup \{ n:S_n \le t\}$ . The counting process $N$ is called a renewal process. The key observation is that $\tau_t$ and $N_t$ are related by $\tau_t = N_t + 1$ . (Note that thus $N_t + 1$ is a stopping time for the $X_i$ .) Thus, $S_{\tau _t } = S_{N_t + 1}$ . This corresponds to $Y_\tau$ of the original question, upon letting $t=K$ . However, in accordance with the common notation used in renewal theory, we shall use $Y$ for the following random variable: we define $Y_t = S_{N_t + 1} - t$ . The random variable $Y_t$ is called the excess at $t$ of the renewal process $N$ . Thus, $Y_t = S_{\tau _t } - t$ , and so (by letting $t=K$ ) this corresponds to the random variable denoted $Y_\tau - K$ in the original question. Hence, as it turns out, the OP actually considered the expectation of the excess at $K$ of a renewal process with inter-arrival times distributed according to the distribution of the $X_i$ .

Finally, here is some useful link concerning renewal theory, which is very relevant to this answer.

Answer by Shai Covo for limit of definite integral as $N \to \infty$

2010-12-19T21:33:18Z

Denote by $I$ your integral. Then, $I = e^N \int_0^1 {x^{N - 1} e^{ - xN} \,{\rm d}x} = \frac{{\Gamma (N)e^N }}{{N^N }}\int_0^N {\frac{{x^{N - 1} e^{ - x} }}{{\Gamma (N)}}\,{\rm d}x}.$ Now, if $X_1,\ldots,X_N$ are independent and identically distributed exponential(1) random variables, then their sum $X_1 + \cdots + X_N$ has gamma density $x^{N-1}e^{-x}/\Gamma(N)$ , $x>0$ . Thus, the last integral above is equal to ${\rm P}(X_1 + \cdots + X_N \leq N)$ , or equivalently to ${\rm P}(X_1 + \cdots + X_N - N \leq 0)$ . By the central limit theorem, ${\rm P}(X_1 + \cdots + X_N - N \leq 0) \to 1/2$ as $N \to \infty$ (since the $X_i$ have expectation equal to $1$ ). So, using $N^N \sim N!e^N /\sqrt {2\pi N} $ , we get $ I \sim \sqrt {\frac{\pi }{2}} \frac{1}{{\sqrt N }}.$

Answer by Shai Covo for Continuous or analytic functions with this property of sinc function

2010-12-12T06:36:48Z

Perhaps the most simple example of an analytic function $f$ , other than $0$, satisfying all three equalities is $f(x) = \sin(\pi x)/ (\pi x)$ . This is verified immediately by a change of variable $x'=\pi x$ , giving $\int_{ - \infty }^\infty {f(x)\,{\rm d}x} = \int_{ - \infty }^\infty {f^2 (x)\,{\rm d}x} = 1 = \sum\nolimits_n {f(n)} = \sum\nolimits_n {f^2 (n)}$ .

EDIT: TCL provided a (very interesting) generalization.

Answer by Shai Covo for Continuous or analytic functions with this property of sinc function

2010-12-10T01:23:52Z

Here's a simple example of a smooth (rather than analytic) function $f$ satisfying the conditions. Define $f(x) = \exp [ - ax^2 /(1 - x^2 ) + bx]$ if $x \in (-1,1)$ , and $f(x)=0$ otherwise. Here $a$ and $b$ are certain positive constants, to be evaluated below. That the function $f$ is smooth (i.e. infinitely differentiable) should be clear by comparison with the function $\exp(-1/x^2)$ (where the point $0$ corresponds to the points $\pm 1$ ). Now, since $f(0)=1$ and $f(n)=0$ for any integer $n \neq 0$ , it remains to have $\int_{ - 1}^1 {f(x)\,{\rm d}x} = \int_{ - 1}^1 {f^2 (x)\,{\rm d}x} = 1$ . Comparing the logs of $f$ and $f^2$ , it is not surprising that there indeed exist $a$ and $b$ satisfying the two integral conditions. With some effort, one is likely to be able to prove this rigorously. However, for our purposes it is enough to be convinced by numerical results. Well, it is easy to get close to a solution. For example, letting $a=3.25247$ and $b=2.08761361$ gives $\int_{ - 1}^1 {f(x)\,{\rm d}x} \approx 0.999999999149$ , $\int_{ - 1}^1 {f^2 (x)\,{\rm d}x} \approx 1.000000136$ .

Answer by Shai Covo for Is it preferable to use a geometric mean with logarithmically distributed samples?

2010-12-02T02:09:10Z

As a particular case of the following proposition, the geometric mean will converge (almost surely) to a number smaller than the actual mean.

Proposition. Suppose that $X_1,X_2,\ldots$ is a sequence of i.i.d. non-constant random variables taking values in $[1,\infty)$ and having a finite mean ${\rm E}(X_1) < \infty$ (this is the case for the logarithmic distribution). Then, almost surely, $\mathop {\lim }\limits_{n \to \infty } \sqrt[n]{{X_1 \cdots X_n }} < {\rm E}(X_1).$

Proof. Define $Y_i = \ln X_i$ . Thus, $(Y_i)$ is a sequence of i.i.d. rv's taking values in $[0,\infty)$ and having a finite mean ${\rm E}(Y_1) (< {\rm E}(X_1))$ . Then, by the strong law of large numbers, $\bar Y_n \to {\rm E}(Y_1)$ almost surely, where $\bar Y_n = \frac{1}{n}\sum\nolimits_{i = 1}^n {Y_i } $ . Hence, since $\sqrt[n]{{X_1 \cdots X_n }} = \exp (\bar Y_n )$ , almost surely $\sqrt[n]{{X_1 \cdots X_n }}$ converges to $\exp({\rm E}(Y_1))$ . By Jensen's inequality, since the function $e^x$ is strictly convex and $Y_1$ is non-constant, $\exp({\rm E}(Y_1)) < {\rm E}[\exp(Y_1)]$ . Noting that the right-hand side is ${\rm E}(X_1)$ , the proposition is proved.

EDIT: In fact, the statement of this proposition continues to hold even if we replace $[1,\infty)$ with $(0,\infty)$. I note this since the OP might be interested in a distribution other than the logarithmic distribution (which is supported on the positive integers).

Answer by Shai Covo for When is the function of a median closer to the median of the function than the mean of the function is to the function of the mean?

2010-11-25T14:14:25Z

REVISED ANSWER:

Lemma 1. If $f$ is not convex, then we can find a random variable $X$ such that $\mu (f(X)) - f(\mu (X)) < m(f(X)) - f(m(X))$ .

Proof. If $f$ is not convex, then, by definition, there exist two points $a \neq b$ and a $p \in (0,1)$ such that $f(pa + (1-p)b) > pf(a) + (1-p)f(b)$ . If this inequality holds for $p = 1/2$ but not for $p = 3/4$ , then it also holds with $p = 2/3$ and $a$ replaced by $3a/4 + b/4$ . Thus, WLOG we can assume that $f(pa + (1-p)b) > pf(a) + (1-p)f(b)$ for some $a \neq b$ and $p > 1/2$ . Define random variable $X$ as follows: ${\rm P}(X = a) = p$ , ${\rm P}(X = b) = 1-p$ . Then, $m(X) = a$ and $m(f(X)) = f(a)$ , so $m(f(X)) - f(m(X)) = 0$ . The lemma now follows from $\mu (f(X)) - f(\mu (X)) = pf(a) + (1-p)f(b) - f(pa + (1-p)b) < 0$ .

Lemma 2. If for some $a < b < c$ , $f(a) = f(c) > f(b)$ , then we can find a random variable $X$ such that $\mu (f(X)) - f(\mu (X)) < m(f(X)) - f(m(X))$ .

Proof. Fix $\varepsilon > 0$ sufficiently small. Define random variable $X$ as follows: ${\rm P}(X=b) = 1/2 - \varepsilon$ , ${\rm P}(X=a) = (1/2 + \varepsilon)(c - b)/(c - a)$ , ${\rm P}(X=c) = (1/2 + \varepsilon)(b - a)/(c - a)$ . Then, $\mu(X)=m(X)=b$ , and ${\rm P}[f(X) = f(b)] = 1/2 - \varepsilon$ , ${\rm P}[f(X) = f(a)]= 1/2 + \varepsilon$ . The lemma now follows from $\mu(f(X)) = (1/2 - \varepsilon)f(b) + (1/2 + \varepsilon)f(a) < f(a) = m(f(X))$ .

From Lemmas 1 and 2, we conclude:

Corollary. The inequality $\mu (f(X)) - f(\mu (X)) \geq m(f(X)) - f(m(X))$ holds for any (integrable) random variable $X$ only if $f$ is convex and monotone.

EDIT: A somewhat trivial extension of this corollary ("if part").

Suppose that $f$ is a strictly monotone convex function, defined on an interval $I$ containing the range of an integrable rv $X$ . If $m(X)$ is unique, then so is $m(f(X))$ , and it holds $\mu (f(X)) - f(\mu (X)) \geq m(f(X)) - f(m(X)) = 0$ .

Proof. By Jensen's inequality, $\mu (f(X)) - f(\mu (X)) \geq 0$ . So, it remains to show that $m(f(X)) = f(m(X))$ . Suppose that $\tilde m \in f(I)$ is a median of $f(X)$ . Then, by definition, ${\rm P}(f(X) \leq \tilde m) \geq 1/2$ and ${\rm P}(f(X) \geq \tilde m) \geq 1/2$ . Taking inverses shows that $f^{-1}(\tilde m)$ is a median of $X$ . Thus, $f^{-1}(\tilde m) = m(X)$ , and the assertion follows.

Answer by Shai Covo for Brownian bridge interpreted as Brownian motion on the circle

2010-11-22T22:19:22Z

First of all, a Brownian motion can be indexed by a quite arbitrary parameter set (so-called set-indexed Brownian motion).

Now to your question. There is a natural extension to two-parameters of the ordinary Brownian motion on $\mathbb{R}_+$ , namely the Brownian sheet, commonly denoted by $W$ and indexed by $\mathbb{R}_ + ^2 = \{ (s,t):s \ge 0,t \ge 0\} $ . A very good textbook on this topic (in a general setting) is "An Introduction to Continuity, Extrema, and Related Topics for General Gaussian Processes"; the relevant pages (6-7) are available online. Now, restricting a Brownian sheet to parametrized curves in $\mathbb{R}^2_+$ results in one-parameter processes, which are continuous centered Gaussian processes. A prominent example is the one-parameter process $A = \{A_t: 0 \leq t \leq 1\}$ defined by $A_t = W_{t,1-t}$ . The process $A$ is in fact a Brownian bridge. So, this is an example of a Brownian bridge which is "a kind of Brownian motion indexed by points on" the segment connecting $(0,1)$ and $(1,0)$ . However, this is essentially the only way to get a Brownian bridge by restricting a Brownian sheet to some parametrized curve in $\mathbb{R}^2_+$ . Nevertheless, a Brownian bridge which is "a kind of Brownian motion indexed by points on the circle" might be obtained as follows. The Brownian sheet is defined, for $s,t \geq 0$ , by $W_{s,t} = W((0,s] \times (0,t])$ , where $W(A)$ , $A \in \mathcal{B}(\mathbb{R}_ + ^2 )$ is the Gaussian white noise based on Lebesgue measure; see pages 6-7 in the aforementioned book. Now, letting the underlying measure $\nu$ in $(\mathbb{R}_ + ^2,\mathcal{B}(\mathbb{R}_ + ^2 ),\nu)$ be a ( $\sigma$ -finite) measure different from Lebesgue measure, the process $\tilde W$ defined by $\tilde W_{s,t} = W((0,s] \times (0,t])$ , where $W$ is a Gaussian white noise based on $\nu$ , is no longer a Brownian sheet; it is very likely that one can find $\nu$ such that the process $\tilde A = \{\tilde A_t: 0 \leq t \leq \pi/2\}$ defined by $\tilde A_t = \tilde W_{\sin t, \cos t}$ is a Brownian bridge (which is "a kind of Brownian motion indexed by points on the circle"). I can guide you how to find the suitable $\nu$, if you wish.

Answer by Shai Covo for How to sample pairwise independent gaussians

2010-11-16T17:17:36Z

Here is the answer I promised in my last comment.

Instead of considering ${\rm N}(0,1)$ variables, we may consider uniform$[0,1)$ variables. Indeed, if $Z_i$ are i.i.d. ${\rm N}(0,1)$ variables, then, with $\Phi(\cdot)$ denoting the ${\rm N}(0,1)$ distribution function, $U_i := \Phi (Z_i)$ are i.i.d. uniform$[0,1)$ variables. In turn, if $\tilde U_i$ are pairwise independent uniform$[0,1)$ variables, then $\tilde Z_i := \Phi^{-1} (\tilde U_i)$ are pairwise independent ${\rm N}(0,1)$ variables.

The rest of this answer is based on the recent paper "Recycling physical random numbers", available at 1 or 2. Henceforth, we use the same letters as in that paper. Suppose that $U_1,\ldots,U_n$ are independent uniform$[0,1)$ variables. Fix $2 \leq m \leq n$, and define $N_m = {n \choose m}$. Now let $X_i$, for $i = 1,\ldots,N_m$, comprise all $N_m$ distinct sums of the form $U_{r_1 } \oplus U_{r_2 } \oplus \cdots \oplus U_{r_m }$, for $1 \le r_1 < r_2 < \cdots < r_m \le n$. Here $U_{r_1 } \oplus U_{r_2 } \oplus \cdots \oplus U_{r_m }$ is the sum modulo $1$ of the $U_{r_i}$, given explicitly by $$ U_{r_1 } \oplus U_{r_2 } \oplus \cdots \oplus U_{r_m } = U_{r_1 } + U_{r_2 } + \cdots + U_{r_m } - \left\lfloor {U_{r_1 } + U_{r_2 } + \cdots + U_{r_m } } \right\rfloor , $$ where $\left\lfloor \cdot \right\rfloor$ is the floor function. Then, the $X_i$ are pairwise independent uniform$[0,1)$ variables. In particular, by letting $m=2$, we can efficiently construct $n(n-1)/2$ pairwise independent uniform variables from $n$ independent ones.

Finally, for general purposes it might be worth stating the following simple fact (Proposition 2 in the aforementioned paper). For $N \geq 2$, let $Y_1,\ldots,Y_N$ be pairwise independent random variables with common mean $\mu$ and common variance $\sigma^2 < \infty$. Define $\bar Y = \frac{1}{N}\sum\nolimits_{i = 1}^N {Y_i }$ and $s^2 = \frac{1}{{N - 1}}\sum\nolimits_{i = 1}^N {(Y_i - \bar Y)^2 }$. Then, ${\rm E}(\bar Y) = \mu$, ${\rm Var}(\bar Y) = \sigma^2/N$, and ${\rm E}(s^2) = \sigma^2$. Combined with the previous paragraph, a straightforward implication is that for a square-integrable function $f$ defined on $[0,1)$, we can approximate the integral $\mu = \int_{[0,1)} {f(x)\,{\rm d}x}$ using a modest number $n$ of independent random inputs. Indeed, note that $n$ independent random inputs can be used to get unbiased Monte Carlo estimates for $\mu$ with the same variance as with $N_m = {n \choose m}$ independent random inputs.

Another special property of the exponential function?

2010-11-09T03:43:57Z

For $x>0$, define $\tilde f(x) = \sum\limits_{k = 0}^\infty {\frac{{(x - k) ^k }}{{k!}} {\bf 1}(x>k)}$ , where ${\bf 1}$ is the indicator function. I know (actually, proved) that $\tilde f(x)$ is asymptotically $a {\rm e}^{bx} $ as $x \to \infty$ , for certain $a>0$ and $0<b<1$ . So, with $f(x)={\rm e}^x$, we have $\sum\limits_{k = 0}^\infty {f^{(k)} (0)\frac{{(x - k)^k }}{{k!}}{\bf 1}(x > k)} \sim af(bx)$ . Are there other (non-trivial, but preferably simple) examples of functions $f$ for which such asymptotic equality holds?

Since the result for $f(x)={\rm e}^x$ is interesting in its own right, it is worth noting that, in fact, $\tilde f(x) \sim \frac{1}{{1 + b}}{\rm e}^{bx}$ , where $b = 0.567143...$ is the solution $b \in (0,1)$ of $b{\rm e}^b = 1$. Moreover, the approximation $\tilde f(x) \approx \frac{1}{{1 + b}}{\rm e}^{bx}$ is most impressive, and is valid even for relatively small $x$ values; for example, $\tilde f(5) = 10.875$ while $\frac{1}{{1 + b}}{\rm e}^{5b} \approx 10.87495$ (for large $x$ values the approximation is extremely accurate).

Comment by Shai Covo

2011-03-17T20:27:00Z

Some useful references are given in mathoverflow.net/questions/30619/… (see the accepted answer; you are not going to find something better than approximations).

Comment by Shai Covo

2011-03-17T16:54:59Z

Can you solve the problem if $X_1$ and $X_2$ are independent?

Comment by Shai Covo

2011-03-16T14:44:02Z

Comment by Shai Covo

2011-03-16T14:43:24Z

Comment by Shai Covo

2011-03-15T20:58:15Z

@mr.gondolier: For the case $\alpha = 1/2$, I suggest writing the series as $(a_1 - a_2) + (a_3 - a_4) + \cdots$, applying the mean value theorem, and considering math.stackexchange.com/questions/27123/….

Comment by Shai Covo

2011-03-14T21:20:46Z

It is stated in the last paper that "There is no universal agreement in the literature on terminology concerning the Markov properties of random fields; see..."

Comment by Shai Covo

2011-03-14T21:14:07Z

References 7,8 in that paper may be useful in order to understand the Introduction (I hope you have access to at least one of them). Moreover, you can find relevant papers dealing with the "sharp Markov property" for random fields, for example faculty.sbc.edu/robeva/…

Comment by Shai Covo

2011-03-14T00:51:54Z

See mathkb.com/Uwe/Forum.aspx/symbolic-math/936/…

Comment by Shai Covo

2011-03-13T09:53:29Z

@mapthemoodle: You can get an answer at math.stackexchange.com

Comment by Shai Covo

2011-03-06T18:06:57Z

On the other hand, deriving an elementary expression for ${\rm E}|B_s B_t|$, $0 < s < t$, is not so easy.

Comment by Shai Covo

2011-02-24T16:28:49Z

Thank you. For $\lim _{t \to 0 \pm } {\rm corr}(X,e^{tX} ) = \pm 1$, I considered the fact that ${\rm corr}(X,1 + tX) = \frac{t}{{|t|}}$.

Comment by Shai Covo

2011-02-23T17:52:00Z

@Albert: I don't have access to that article.

Comment by Shai Covo

2011-02-23T15:59:04Z

Apparently, the OP is referring to the following paper (access to the content is restricted to subscribers): springerlink.com/content/3tru5ectxdmb9p67

Comment by Shai Covo

2011-02-15T15:47:01Z

@Steve and all: You are welcome to edit my answer.

Comment by Shai Covo

2011-02-15T06:37:55Z

@Mike: Great observation, thanks! I'll edit the answer soon.