1
$\begingroup$

Let

  • $f\in C^3(\mathbb R)$ with $f>0$ and $$\int f(x)\:{\rm d}x=1$$
  • $g:=\ln f$ (and assume $g'$ is Lipschitz continuous)
  • $n\in\mathbb N$, $$s(x,y):=\sum_{i=1}^n\left(g(y_i)-g(x_i)\right)$$ and $$h(x,y):=\min\left(1,e^{s(x,\:y)}\right)$$ for $x,y\in\mathbb R^n$
  • $x\in\mathbb R^n$ and $Y$ be a $\mathbb R^n$-valued normally distributed random variable on a probability space $(\Omega,\mathcal A,\operatorname P)$ with mean vector $x$ and covariance matrix $\sigma I_n$ for some $\sigma>0$ ($I_n$ denoting the $n\times n$ identity matrix)

I want to make the following argumentation rigorous: By Taylor's theorem, \begin{equation}\begin{split}h(x,Y)-h(x,(x_1,Y_2,\ldots,Y_n))&=\frac{\partial h}{\partial y_1}(x,(x_1,Y_2,\ldots,Y_n))(Y_1-x_1)\\&+\frac12\frac{\partial^2h}{\partial y_1^2}(x,(Z_1,Y_2,\ldots,Y_n))(Y_1-x_1)^2\end{split}\tag1\end{equation} for some real-valued random variable $Z_1$ with $Z_1\in[\min(x_1,Y_1),\max(x_1,Y_1)]$. Thus, \begin{equation}\begin{split}\left.\operatorname E\left[h(x,(y_1,Y_2,\ldots,Y_n))\right]\right|_{y_1\:=\:Y_1}&=\operatorname E\left[\min\left(1,e^A\right)\right]+g'(x_1)\operatorname E\left[1_{\left\{\:A\:<\:0\:\right\}}e^A\right](Y_1-x_1)\\&+\frac12(g''(Z_1)+\left|g'(Z_1)\right|^2)\left.\operatorname E\left[1_{\left\{\:B\:<\:0\:\right\}}e^B\right]\right|_{z_1\:=\:Z_1}(Y_1-x_1)^2.\end{split}\tag2\end{equation} Above, I wrote $$A:=\sum_{i=2}^n(g(Y_i)-g(x_i))$$ and $$B:=g(z_1)-g(x_1)+\sum_{i=2}^n(g(Y_i)-g(x_i))$$ in order to make the equation more readable (you need to replace them where they occur).

Question 1: There are two issues: The first one is that $(x,y)\mapsto\min(x,y)$ is partially differentiable in both arguments except on the diagonal $\Delta_2:=\left\{(x,y)\in\mathbb R^2:x=y\right\}$. Are we able to conclude the existence of $Z_1$ anyway? Note that $$\frac{\partial h}{\partial y_1}(x,y)=\begin{cases}\displaystyle g'(y_1)e^{s(x,\:y)}&\text{, if }s(x,y)<0\\0&\text{, if }s(x,y)>0\end{cases}\tag3$$ and $$\frac{\partial^2h}{\partial y_1^2}(x,y)=\begin{cases}\displaystyle(g''(y_1)+|g'(y_1)|^2)e^{s(x,\:y)}&\text{, if }s(x,y)<0\\0&\text{, if }s(x,y)>0\end{cases}\tag4$$ for all $y\in\mathbb R^n$.

Question 2: The second issue is the case $s(x,y)=0$. In order for $(3)$ to hold, we need to show that the probability of the corresponding event is $0$ (this seems to be related to the question whether the set on which the occurring function is not differentiable has Lebesgue measure $0$; and it's clear that $\Delta$ has Lebesgue measure $0$). How can we do that?

While it's clear that $h$ is partially differentiable with respect to the second variable except on a countable set, it is not clear to me why $h$ is even twice differentiable with respect to the second variable except on a set (at least) of Lebesgue measure $0$ (see this related question).

EDIT: Please take note of this related question which might yield a solution for question 2.

$\endgroup$
  • $\begingroup$ Maybe we need to use a version of the Taylor formula with integral remainder which is valid when the derivative in the integral is only defined in a weak sense. In that case, it would be enough to show that $h$ has a weak second partial derivative. $\endgroup$ – 0xbadf00d May 5 at 7:54
1
$\begingroup$

Obviously in the case that $n=1$, we have $s(x,x)=0$ and so if $f'(x) \neq 0$ then $\frac{\partial h}{\partial y_1}$ doesn't exist at $(x,x)$. So I will assume that $n \geq 2$.

Answer to Question 1. The random variable $Z_1$ will pretty much never exist. As an example, take $f$ to be the pdf of a normal distribution of standard deviation $1$, take $n=2$, and take $(x_1,x_2)=(1,0)$; then on the event $\left\{Y_2 \in (-1,1), f(Y_1)>\frac{f(1)f(0)}{f(Y_2)}\right\}$, we have that the LHS of (1) is strictly positive, but the RHS of (1) is nonpositive when $Z_1$ is replaced by any number between $x_1$ and $Y_1$.

Answer to Question 2. By "the corresponding event", I guess you mean the event $\{ s(x,(x_1,Y_2,\ldots,Y_n)) = 0 \}$? This is the same as the event $$ \left\{ \sum_{i=2}^n g(Y_i) = c \right\} $$ where $c=\sum_{i=2}^n g(x_i)$. If there exist $p_1,\ldots,p_{n-1} \in \mathbb{R}$ such that $\sum_{i=1}^{n-1} p_i = c$ and $g^{-1}(\{p_i\})$ has positive Lebesgue measure for each $i$, then clearly this event has positive probability. Otherwise, this event has zero probability:

Theorem. Let $X_1,\ldots,X_N$ be independent random variables each with law equivalent to the Lebesgue measure, and fix any $c \in \mathbb{R}$. Let $g \colon \mathbb{R} \to \mathbb{R}$ be a measurable function and suppose the event $$ E := \left\{ \sum_{i=1}^N g(X_i) = c \right\} $$ has positive probability. Then there exists $(p_1,\ldots,p_N) \in \mathbb{R}^N$ such that $\sum_{i=1}^N p_i = c$ and $g^{-1}(\{p_i\})$ has positive Lebesgue measure for each $i$.

Proof: The statement is obvious for $N=1$ so assume $N \geq 2$. Let $S:=\{p \in \mathbb{R} : \mathrm{Leb}(g^{-1}(\{p\}))>0 \}$. Obviously $S$ is at most countable. We recursively construct $(p_1,\ldots,p_i) \in S^i$, $i \in \{1,\ldots,N-1\}$, such that the event $$ \left\{ \sum_{j=i+1}^N g(X_j) \ = \ c - \sum_{j=1}^i p_j \right\} $$ has positive probability. For $i=N-1$ we then simply define $p_N=c-\sum_{j=1}^{N-1} p_j$, and we are done.

For $i=1$: Fubini's theorem gives that $$ \mathbb{P}\left( c - \sum_{j=2}^N g(X_j) \in S \right) > 0. $$ So since $S$ is at most countable, there exists $p_1 \in S$ such that $$ \mathbb{P}\left( \sum_{j=2}^N g(X_j) = c - p_1 \right) > 0. $$

Now suppose we have $i \in \{1,\ldots,N-2\}$ and $(p_1,\ldots,p_i) \in S^i$ such that the event $$ \left\{ \sum_{j=i+1}^N g(X_j) \ = \ c - \sum_{j=1}^i p_j \right\} $$ has positive probability. Then Fubini's theorem gives that $$ \mathbb{P}\left( c - \left( \sum_{j=1}^i p_j \ \; + \, \sum_{j=i+2}^N g(X_j) \right) \in S \right) > 0. $$ So since $S$ is at most countable, there exists $p_{i+1} \in S$ such that $$ \mathbb{P}\left( \sum_{j=i+2}^N g(X_j) \ = \ c - \sum_{j=1}^{i+1} p_j \right) > 0. $$ So we are done. $\square$

$\endgroup$
  • $\begingroup$ I know it would probably take a long time to go through the details of this answer, so at this late stage in the bounty period, please don't worry about awarding the bounty if you don't get time before the end of the "grace period" to check whether you are happy with my answer. $\endgroup$ – Julian Newman May 11 at 2:13
  • $\begingroup$ Regarding question 1: Does the situation change if $x$ is replaced by a $\mathbb R^d$-valued random variable $X\sim\pi_d\lambda^d$, where $\pi_d(x):=\prod_{i=1}^df(x_i)$ for $x\in\mathbb R^d$ and $\lambda^d$ is the Lebesgue measure on $\mathcal B(\mathbb R^d)$. In that case we would assume that $\operatorname P[Y\in B\mid X]=\mathcal N_d(X,\sigma_d^2I_d)(B)$ for all $B\in\mathcal B(\mathbb R^d)$. $\endgroup$ – 0xbadf00d May 11 at 5:49
  • $\begingroup$ In what you've written, $X$ looks like an $\mathbb{R}$-valued random variable, not an $\mathbb{R}^d$-valued random variable. But in any case, trying to get Taylor's theorem (in the sense of a "mean-value form of the remainder") for a function with a sharp corner in it seems hopeless, unless you have some way of guaranteeing that $X_1$ and $Y_1$ always occur on the same side of the sharp corner. $\endgroup$ – Julian Newman May 11 at 13:45
  • $\begingroup$ Why does it look like an $\mathbb R$-valued random variable? $\mathcal N_d$ denotes the $d$-dimensional normal distribution. So the first parameter is from $\mathbb R^d$. (And note that $(x,B)\mapsto\mathcal N_d(x,\sigma_d^2I_d)(B)$ is a Markov kernel). To give all the questions you're responding to a context: I'm trying to understand the proof of Lemma A.3 in the paper Optimal scaling for partially updating MCMC algorithms. The application of Taylor's theorem seems to be wrong (for the reasons discussed so far) and I'm searching for a way to fix it. $\endgroup$ – 0xbadf00d May 11 at 15:50
  • $\begingroup$ Maybe I've missed a crucial assumption. Please take a look at the following question, where I've chosen a particular $f$ satisfying the requirements I've got in mind: math.stackexchange.com/q/3281130/47771. $\endgroup$ – 0xbadf00d Jul 2 at 19:05

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.