1
$\begingroup$

Let $(\epsilon_t)_t$ be a sequence of iid random variables, distributed according to the density $f:\mathbb{R}\to (0,\infty)$ and $$ x_t = q( \theta^\star, x_1,x_2, \ldots, x_{t-1}) + \epsilon_t \,. $$ Assume that the derivative of $q$ with respect to $\theta$ is bounded from below, i.e. for all $\theta, x$ $$ \frac{\partial q(\theta, (x_r)_{r<t})}{\partial \theta} \geq c > 0\,. $$ The posterior likelihood is defined as $$ L_t(\theta) = \sum_{s=1}^t \log f \Big(x_s-q(\theta, (x_r)_{r<t})\Big) + \log \pi_0(\theta)\,, $$ where the prior density $\pi_0:\mathbb{R} \to (0,\infty)$ is strictly positive.

Question: Are those assumptions sufficient to conclude that the posterior $$ \pi_t(\theta) = \frac{e^{L_t(\theta)}}{\int e^{L_t(z)} dz} $$ converges almost surely to a dirac measure on $\theta^\star$? How could I show this? What would be a good reference?

I know that Bernstein von-Mises Theorem yields this type of result, but I could not find a version which is general enough to cover this case.

Thanks a lot!

$\endgroup$
1
$\begingroup$

Probably not true just with the assumptions you have there. It seems to accommodate a model in which $\theta$ is the starting point of a random walk. That would only require $q = \theta + x_1 + ... + x_t$. It is known and easy to believe that in this case you cannot estimate $\theta $ consistently, and I would think that therefore the posterior should not collapse to a point mass. You can calculate the posterior explicitly in case $\theta, \epsilon_t$ are i.i.d. normal see that that is true.

$\endgroup$
  • $\begingroup$ Do you know a reference arguing that $\theta$ can not be estimated consistently in this case? Thank you. $\endgroup$ – Peter Nov 14 '16 at 22:26
  • $\begingroup$ Do you know of a restriction on $q$ that ensures convergence (allowing for dependence $x_1, \ldots, x_{t-1}$)? $\endgroup$ – Peter Nov 14 '16 at 23:31
  • $\begingroup$ I regret that I don't know a reference, but the technique is to show that the measures starting from different $\theta$ are absolutely comntinuous up to and including infinity. Then, you can't have a function that is $\theta_1$ with probability 1 under $P_{\theta_1}$ and $\theta_2$ under $P_{\theta_2}$. I do not know any general conditions in this framework, but is there a specific problem you are working on ? $\endgroup$ – user83457 Nov 15 '16 at 8:55
  • $\begingroup$ Thanks again Michael! I am interested in the case where $q(\theta,m(L_{t-1}))$ is a function of $\theta$ and a one dimensional summary statistic $m(L_{t-1}) \in \mathbb{R}$ of the log-likelihood in the previous period. I was able to prove convergence under somewhat very restrictive restrictions on $f$. do you know any result that might cover this more specific setup? $\endgroup$ – Peter Nov 21 '16 at 0:12
  • $\begingroup$ No, because it has the same problem, in the case I outlined $X_1$ is the sufficient statistic, and that is part of the problem because the first observation has all the information you ever get about the parameter. $\endgroup$ – user83457 Nov 23 '16 at 12:15

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.