Skip to main content
added 23 characters in body
Source Link
Iosif Pinelis
  • 127.7k
  • 8
  • 107
  • 229

Let us prove the desired conclusion generally, without assuming that the random variables $\xi$ and $\eta$ are discrete. Let $g\colon\mathbb R\to\mathbb R$ be any strictly increasing strictly convex differentiable function such that $|g(x)|\le1+|x|$ for all real $x$, so that $Eg(\xi)$$|g'|\le1$ and $Eg(\eta)$$Eg(\xi),Eg(\eta)$ exist in $\mathbb R$. (For instance, one may take $g(x):=E(x+Z)_+=\int_{-x}^\infty(x+z)\varphi(z)\,dz$, so that $0<g(x)\le|x|+EZ_+\le|x|+1$ and $g'(x)=\int_{-x}^\infty\varphi(z)\,dz=\int_{-\infty}^x \varphi(z)\,dz$, where $Z\sim N(0,1)$ and $\varphi$ is the pdf of $Z$.)

By the convexity of $g$, \begin{equation} g(\xi)\ge g(E_\eta\xi)+g'(E_\eta\xi)(\xi-E_\eta\xi), \tag{1} \end{equation} where $E_\eta\xi:=E(\xi|\eta)$. Moreover, by the strict convexity of $g$, inequality (1) is strict on the event $\{\xi\ne E_\eta\xi\}$. Taking the expectations of both sides of (1), we have \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi)+EE_\eta g'(E_\eta\xi)(\xi-E_\eta\xi) =Eg(E_\eta\xi)+Eg'(E_\eta\xi)E_\eta (\xi-E_\eta\xi) =Eg(E_\eta\xi), \end{equation} so that we have the conditional Jensen inequality \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi), \tag{2} \end{equation} and this inequality is strict unless $\xi=E_\eta\xi$ almost surely (a.s.). Similarly, \begin{equation} Eg(\eta)\ge Eg(E_\xi\eta), \tag{3} \end{equation} and this inequality is strict unless $\eta=E_\xi\eta$ a.s.

Using now (2), the conditions $E_\eta\xi\ge\eta$ and $E_\xi\eta\ge\xi$ given in the OP, the condition that $g$ is increasing, and (3), we have \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi)\ge Eg(\eta)\ge Eg(E_\xi\eta)\ge Eg(\xi), \tag{4} \end{equation} and (since $g$ is strictly increasing) the 2nd and 4th inequalities here are strict unless, respectively, $E_\eta\xi=\eta$ a.s. and $E_\xi\eta=\xi$ a.s. However, all the inequalities in (4) must be the equalities. It follows that $\xi=E_\eta\xi=\eta$ a.s., whence $\xi=\eta$ a.s., as desired.

Let us prove the desired conclusion generally, without assuming that the random variables $\xi$ and $\eta$ are discrete. Let $g\colon\mathbb R\to\mathbb R$ be any strictly increasing strictly convex function such that $|g(x)|\le1+|x|$ for all real $x$, so that $Eg(\xi)$ and $Eg(\eta)$ exist in $\mathbb R$. (For instance, one may take $g(x):=E(x+Z)_+=\int_{-x}^\infty(x+z)\varphi(z)\,dz$, so that $0<g(x)\le|x|+EZ_+\le|x|+1$ and $g'(x)=\int_{-x}^\infty\varphi(z)\,dz=\int_{-\infty}^x \varphi(z)\,dz$, where $Z\sim N(0,1)$ and $\varphi$ is the pdf of $Z$.)

By the convexity of $g$, \begin{equation} g(\xi)\ge g(E_\eta\xi)+g'(E_\eta\xi)(\xi-E_\eta\xi), \tag{1} \end{equation} where $E_\eta\xi:=E(\xi|\eta)$. Moreover, by the strict convexity of $g$, inequality (1) is strict on the event $\{\xi\ne E_\eta\xi\}$. Taking the expectations of both sides of (1), we have \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi)+EE_\eta g'(E_\eta\xi)(\xi-E_\eta\xi) =Eg(E_\eta\xi)+Eg'(E_\eta\xi)E_\eta (\xi-E_\eta\xi) =Eg(E_\eta\xi), \end{equation} so that we have the conditional Jensen inequality \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi), \tag{2} \end{equation} and this inequality is strict unless $\xi=E_\eta\xi$ almost surely (a.s.). Similarly, \begin{equation} Eg(\eta)\ge Eg(E_\xi\eta), \tag{3} \end{equation} and this inequality is strict unless $\eta=E_\xi\eta$ a.s.

Using now (2), the conditions $E_\eta\xi\ge\eta$ and $E_\xi\eta\ge\xi$ given in the OP, the condition that $g$ is increasing, and (3), we have \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi)\ge Eg(\eta)\ge Eg(E_\xi\eta)\ge Eg(\xi), \tag{4} \end{equation} and (since $g$ is strictly increasing) the 2nd and 4th inequalities here are strict unless, respectively, $E_\eta\xi=\eta$ a.s. and $E_\xi\eta=\xi$ a.s. However, all the inequalities in (4) must be the equalities. It follows that $\xi=E_\eta\xi=\eta$ a.s., whence $\xi=\eta$ a.s., as desired.

Let us prove the desired conclusion generally, without assuming that the random variables $\xi$ and $\eta$ are discrete. Let $g\colon\mathbb R\to\mathbb R$ be any strictly increasing strictly convex differentiable function such that $|g(x)|\le1+|x|$ for all real $x$, so that $|g'|\le1$ and $Eg(\xi),Eg(\eta)$ exist in $\mathbb R$. (For instance, one may take $g(x):=E(x+Z)_+=\int_{-x}^\infty(x+z)\varphi(z)\,dz$, so that $0<g(x)\le|x|+EZ_+\le|x|+1$ and $g'(x)=\int_{-x}^\infty\varphi(z)\,dz=\int_{-\infty}^x \varphi(z)\,dz$, where $Z\sim N(0,1)$ and $\varphi$ is the pdf of $Z$.)

By the convexity of $g$, \begin{equation} g(\xi)\ge g(E_\eta\xi)+g'(E_\eta\xi)(\xi-E_\eta\xi), \tag{1} \end{equation} where $E_\eta\xi:=E(\xi|\eta)$. Moreover, by the strict convexity of $g$, inequality (1) is strict on the event $\{\xi\ne E_\eta\xi\}$. Taking the expectations of both sides of (1), we have \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi)+EE_\eta g'(E_\eta\xi)(\xi-E_\eta\xi) =Eg(E_\eta\xi)+Eg'(E_\eta\xi)E_\eta (\xi-E_\eta\xi) =Eg(E_\eta\xi), \end{equation} so that we have the conditional Jensen inequality \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi), \tag{2} \end{equation} and this inequality is strict unless $\xi=E_\eta\xi$ almost surely (a.s.). Similarly, \begin{equation} Eg(\eta)\ge Eg(E_\xi\eta), \tag{3} \end{equation} and this inequality is strict unless $\eta=E_\xi\eta$ a.s.

Using now (2), the conditions $E_\eta\xi\ge\eta$ and $E_\xi\eta\ge\xi$ given in the OP, the condition that $g$ is increasing, and (3), we have \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi)\ge Eg(\eta)\ge Eg(E_\xi\eta)\ge Eg(\xi), \tag{4} \end{equation} and (since $g$ is strictly increasing) the 2nd and 4th inequalities here are strict unless, respectively, $E_\eta\xi=\eta$ a.s. and $E_\xi\eta=\xi$ a.s. However, all the inequalities in (4) must be the equalities. It follows that $\xi=E_\eta\xi=\eta$ a.s., whence $\xi=\eta$ a.s., as desired.

added 255 characters in body
Source Link
Iosif Pinelis
  • 127.7k
  • 8
  • 107
  • 229

Let $a_1,\dots,a_m$ be allus prove the distinct values ofdesired conclusion generally, without assuming that the random variables $\xi$, and let $b_1,\dots,b_n$ be all the distinct values of $\eta$ are discrete. Let $p_{ij}:=P(\xi=a_i,\eta=b_j)$, $r_i:=P(\xi=a_i)=\sum_j p_{ij}$, $c_j:=P(\eta=b_j)=\sum_i p_{ij}$. Let us interpret the condition "any value of these values are accepted with a non-zero probability" as the condition$g\colon\mathbb R\to\mathbb R$ be any strictly increasing strictly convex function such that $p_{ij}>0$$|g(x)|\le1+|x|$ for all real $i\in[m]:=\{1,\dots,m\}$$x$, so that $Eg(\xi)$ and $j\in[n]$$Eg(\eta)$ exist in $\mathbb R$. Then the conditions $E(\xi|\eta)\ge\eta$ and(For instance, one may take $E(\eta|\xi)\ge\xi$ translate into \begin{equation} \frac1{c_j}\,\sum_i a_i p_{ij}\ge b_j\quad\text{and}\quad \sum_j b_j p_{kj}\ge a_k r_k=:A_k \end{equation} for all$g(x):=E(x+Z)_+=\int_{-x}^\infty(x+z)\varphi(z)\,dz$, so that $k\in[m]$$0<g(x)\le|x|+EZ_+\le|x|+1$ and $j\in[n]$ $g'(x)=\int_{-x}^\infty\varphi(z)\,dz=\int_{-\infty}^x \varphi(z)\,dz$, whencewhere $Z\sim N(0,1)$ and $\varphi$ is the pdf of $Z$.)

By the convexity of $g$, for \begin{equation} M_{ik}:=\frac1{r_i}\sum_j\frac{p_{kj}p_{ij}}{c_j}, \end{equation} \begin{equation} \sum_i A_i M_{ik}=\sum_i a_i \sum_j\frac{p_{kj}p_{ij}}{c_j} =\sum_j\frac{p_{kj}}{c_j}\,\sum_ia_i p_{ij}\ge \sum_j b_j p_{kj}\ge A_k. \tag{*} \end{equation}\begin{equation} g(\xi)\ge g(E_\eta\xi)+g'(E_\eta\xi)(\xi-E_\eta\xi), \tag{1} \end{equation} The matrixwhere $M=[M_{ik}]$ is stochastic, i.e$E_\eta\xi:=E(\xi|\eta)$. Moreover, by the strict convexity of $\sum_k M_{ik}=1$, and so$g$, inequality $\sum_k\sum_i A_i M_{ik}=\sum_i A_i$(1) is strict on the event $\{\xi\ne E_\eta\xi\}$. Hence, allTaking the inequalities inexpectations of both sides of (*1) are equalities, sowe have \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi)+EE_\eta g'(E_\eta\xi)(\xi-E_\eta\xi) =Eg(E_\eta\xi)+Eg'(E_\eta\xi)E_\eta (\xi-E_\eta\xi) =Eg(E_\eta\xi), \end{equation} so that we have the conditional Jensen inequality \begin{equation} \sum_i A_i M_{ik}=A_k \end{equation}\begin{equation} Eg(\xi)\ge Eg(E_\eta\xi), \tag{2} \end{equation} for alland this inequality is strict unless $k$$\xi=E_\eta\xi$ almost surely (a. That iss.). Similarly, $A=[A_1,\dots,A_m]$ \begin{equation} Eg(\eta)\ge Eg(E_\xi\eta), \tag{3} \end{equation} and this inequality is an invariant measure for the stochastic matrixstrict unless $M$ with all strictly positive entries$\eta=E_\xi\eta$ a. Sos.

Using now (2), the vectorconditions $A$ is uniquely determined up to a constant factor. On$E_\eta\xi\ge\eta$ and $E_\xi\eta\ge\xi$ given in the other handOP, it is straightforward to check that the formula $A_i=r_i$ for allcondition that $i$ defines an invariant measure for$g$ is increasing, and $M$. So(3), we have \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi)\ge Eg(\eta)\ge Eg(E_\xi\eta)\ge Eg(\xi), \tag{4} \end{equation} and $a_i=a$ for some real(since $a>0$$g$ is strictly increasing) the 2nd and all $i$. Similarly4th inequalities here are strict unless, $b_j=b$ for some realrespectively, $b>0$$E_\eta\xi=\eta$ a.s. and all $j$$E_\xi\eta=\xi$ a. Nows. However, all the conditions $E(\xi|\eta)\ge\eta$ andinequalities in $E(\eta|\xi)\ge\xi$ imply(4) must be the equalities. It follows that $a\ge b\ge a$$\xi=E_\eta\xi=\eta$ a.s., so thatwhence $\xi=\eta$ a.s., as desired.

Let $a_1,\dots,a_m$ be all the distinct values of $\xi$, and let $b_1,\dots,b_n$ be all the distinct values of $\eta$. Let $p_{ij}:=P(\xi=a_i,\eta=b_j)$, $r_i:=P(\xi=a_i)=\sum_j p_{ij}$, $c_j:=P(\eta=b_j)=\sum_i p_{ij}$. Let us interpret the condition "any value of these values are accepted with a non-zero probability" as the condition that $p_{ij}>0$ for all $i\in[m]:=\{1,\dots,m\}$ and $j\in[n]$. Then the conditions $E(\xi|\eta)\ge\eta$ and $E(\eta|\xi)\ge\xi$ translate into \begin{equation} \frac1{c_j}\,\sum_i a_i p_{ij}\ge b_j\quad\text{and}\quad \sum_j b_j p_{kj}\ge a_k r_k=:A_k \end{equation} for all $k\in[m]$ and $j\in[n]$, whence, for \begin{equation} M_{ik}:=\frac1{r_i}\sum_j\frac{p_{kj}p_{ij}}{c_j}, \end{equation} \begin{equation} \sum_i A_i M_{ik}=\sum_i a_i \sum_j\frac{p_{kj}p_{ij}}{c_j} =\sum_j\frac{p_{kj}}{c_j}\,\sum_ia_i p_{ij}\ge \sum_j b_j p_{kj}\ge A_k. \tag{*} \end{equation} The matrix $M=[M_{ik}]$ is stochastic, i.e., $\sum_k M_{ik}=1$, and so, $\sum_k\sum_i A_i M_{ik}=\sum_i A_i$. Hence, all the inequalities in (*) are equalities, so that \begin{equation} \sum_i A_i M_{ik}=A_k \end{equation} for all $k$. That is, $A=[A_1,\dots,A_m]$ is an invariant measure for the stochastic matrix $M$ with all strictly positive entries. So, the vector $A$ is uniquely determined up to a constant factor. On the other hand, it is straightforward to check that the formula $A_i=r_i$ for all $i$ defines an invariant measure for $M$. So, $a_i=a$ for some real $a>0$ and all $i$. Similarly, $b_j=b$ for some real $b>0$ and all $j$. Now the conditions $E(\xi|\eta)\ge\eta$ and $E(\eta|\xi)\ge\xi$ imply $a\ge b\ge a$, so that $\xi=\eta$.

Let us prove the desired conclusion generally, without assuming that the random variables $\xi$ and $\eta$ are discrete. Let $g\colon\mathbb R\to\mathbb R$ be any strictly increasing strictly convex function such that $|g(x)|\le1+|x|$ for all real $x$, so that $Eg(\xi)$ and $Eg(\eta)$ exist in $\mathbb R$. (For instance, one may take $g(x):=E(x+Z)_+=\int_{-x}^\infty(x+z)\varphi(z)\,dz$, so that $0<g(x)\le|x|+EZ_+\le|x|+1$ and $g'(x)=\int_{-x}^\infty\varphi(z)\,dz=\int_{-\infty}^x \varphi(z)\,dz$, where $Z\sim N(0,1)$ and $\varphi$ is the pdf of $Z$.)

By the convexity of $g$, \begin{equation} g(\xi)\ge g(E_\eta\xi)+g'(E_\eta\xi)(\xi-E_\eta\xi), \tag{1} \end{equation} where $E_\eta\xi:=E(\xi|\eta)$. Moreover, by the strict convexity of $g$, inequality (1) is strict on the event $\{\xi\ne E_\eta\xi\}$. Taking the expectations of both sides of (1), we have \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi)+EE_\eta g'(E_\eta\xi)(\xi-E_\eta\xi) =Eg(E_\eta\xi)+Eg'(E_\eta\xi)E_\eta (\xi-E_\eta\xi) =Eg(E_\eta\xi), \end{equation} so that we have the conditional Jensen inequality \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi), \tag{2} \end{equation} and this inequality is strict unless $\xi=E_\eta\xi$ almost surely (a.s.). Similarly, \begin{equation} Eg(\eta)\ge Eg(E_\xi\eta), \tag{3} \end{equation} and this inequality is strict unless $\eta=E_\xi\eta$ a.s.

Using now (2), the conditions $E_\eta\xi\ge\eta$ and $E_\xi\eta\ge\xi$ given in the OP, the condition that $g$ is increasing, and (3), we have \begin{equation} Eg(\xi)\ge Eg(E_\eta\xi)\ge Eg(\eta)\ge Eg(E_\xi\eta)\ge Eg(\xi), \tag{4} \end{equation} and (since $g$ is strictly increasing) the 2nd and 4th inequalities here are strict unless, respectively, $E_\eta\xi=\eta$ a.s. and $E_\xi\eta=\xi$ a.s. However, all the inequalities in (4) must be the equalities. It follows that $\xi=E_\eta\xi=\eta$ a.s., whence $\xi=\eta$ a.s., as desired.

Source Link
Iosif Pinelis
  • 127.7k
  • 8
  • 107
  • 229

Let $a_1,\dots,a_m$ be all the distinct values of $\xi$, and let $b_1,\dots,b_n$ be all the distinct values of $\eta$. Let $p_{ij}:=P(\xi=a_i,\eta=b_j)$, $r_i:=P(\xi=a_i)=\sum_j p_{ij}$, $c_j:=P(\eta=b_j)=\sum_i p_{ij}$. Let us interpret the condition "any value of these values are accepted with a non-zero probability" as the condition that $p_{ij}>0$ for all $i\in[m]:=\{1,\dots,m\}$ and $j\in[n]$. Then the conditions $E(\xi|\eta)\ge\eta$ and $E(\eta|\xi)\ge\xi$ translate into \begin{equation} \frac1{c_j}\,\sum_i a_i p_{ij}\ge b_j\quad\text{and}\quad \sum_j b_j p_{kj}\ge a_k r_k=:A_k \end{equation} for all $k\in[m]$ and $j\in[n]$, whence, for \begin{equation} M_{ik}:=\frac1{r_i}\sum_j\frac{p_{kj}p_{ij}}{c_j}, \end{equation} \begin{equation} \sum_i A_i M_{ik}=\sum_i a_i \sum_j\frac{p_{kj}p_{ij}}{c_j} =\sum_j\frac{p_{kj}}{c_j}\,\sum_ia_i p_{ij}\ge \sum_j b_j p_{kj}\ge A_k. \tag{*} \end{equation} The matrix $M=[M_{ik}]$ is stochastic, i.e., $\sum_k M_{ik}=1$, and so, $\sum_k\sum_i A_i M_{ik}=\sum_i A_i$. Hence, all the inequalities in (*) are equalities, so that \begin{equation} \sum_i A_i M_{ik}=A_k \end{equation} for all $k$. That is, $A=[A_1,\dots,A_m]$ is an invariant measure for the stochastic matrix $M$ with all strictly positive entries. So, the vector $A$ is uniquely determined up to a constant factor. On the other hand, it is straightforward to check that the formula $A_i=r_i$ for all $i$ defines an invariant measure for $M$. So, $a_i=a$ for some real $a>0$ and all $i$. Similarly, $b_j=b$ for some real $b>0$ and all $j$. Now the conditions $E(\xi|\eta)\ge\eta$ and $E(\eta|\xi)\ge\xi$ imply $a\ge b\ge a$, so that $\xi=\eta$.