Inference for the normal distribution with known variance from multiple clusters

Question

Here's the question:

We have: $q \sim N\left(q_p, \frac{1}{\tau}\right), q_i \sim N\left(q, \frac{1}{\zeta}\right), t_n \sim N\left(0, \frac{1}{\eta}\right)$. Let $$ r_n=\sum_{i=1}^{\theta k_{n}} \dfrac{q_{i}+t_{n}}{\theta k_{n}}$$

$r_n\mid q\sim N\left(q, \frac{1}{w_n}+\frac{1}{\eta}\right)$, $w_n:=\theta k_n \zeta$, $r_n\mid(q,t_n)\sim N(q+t_n,\frac{1}{w_n})$

Let $\vec{r}=[r_1,r_2,\cdots,r_n,\cdots,r_N]$. How to derive the $(\mu, \sigma^2) \text{ in }q+t_n\mid\vec{r}\sim N(\mu,\sigma^2)$?

How to derive the $(\mu,\sigma^2)$ by inference? Notice that the distribution of observation is not equal, and that's where I'm stuck.

Answer 1

We start with our prior belief about $q$, which is normally distributed:

$$ q \sim \mathcal{N}(q_p, \tau^{-1}) $$

For each observation $r_n$, we have this likelihood:

$$ r_n | q \sim \mathcal{N}(q, w_n^{-1} + \eta^{-1}) $$

The $w_n$ term accounts for the varying precision of our observations, where $w_n$ is defined as $\theta k_n \zeta$.

Our goal is to update our belief about $q$ given all these observations $r$. Applying Bayes' theorem, we end up with a posterior distribution that's also Gaussian, thanks to the conjugate prior.

The precision of our posterior distribution is:

$$ \sigma_q^{-2} = \tau + \sum_{n=1}^N \frac{w_n\eta}{w_n+\eta} $$

And the mean is:

$$ \mu_q = \sigma_q^2 \left(\tau q_p + \sum_{n=1}^N \frac{w_n\eta}{w_n+\eta}r_n\right) $$

We're not quite finished, as we need to account for the $t_n$ term. It's independent of our observations and normally distributed with mean $0$ and precision $\eta$.

Adding $q$ and $t_n$ involves summing two independent Gaussian variables. The result is another Gaussian, with a mean equal to the mean of $q$ (since $t_n$ has mean $0$), and a variance that's the sum of the variances of $q$ and $t_n$.

Our final result is:

$$ q+t_n | \vec{r} \sim \mathcal{N}(\mu, \sigma^2) $$

where:

$$ \mu = \sigma_q^2 \left(\tau q_p + \sum_{n=1}^N \frac{w_n\eta}{w_n+\eta}r_n\right) $$

$$ \sigma^2 = \left(\tau + \sum_{n=1}^N \frac{w_n\eta}{w_n+\eta}\right)^{-1} + \eta^{-1} $$