What is the Kullback-Leibler divergence of two Student's T distributions that have been shifted and scaled? That is, $\textrm{D}_{\textrm{KL}}(k_aA + t_a; k_bB + t_b)$ where $A$ and $B$ are Student's T distributions.

If it makes things easier, $A$ could be a Gaussian. (That is, it could have infinite degrees of freedom.)

The motivation behind this question is that the scaled non-central Student's T distribution is the posterior predictive distribution of normally distributed data with unknown mean and variance. Thus, I would like to compare the true distribution $k_aA + t_a$ with the estimate $k_bB + t_b$.

  • $\begingroup$ Just to make sure: what was shifted and scaled is actually the x value. If the pdf of Student's t is f(x), then by kA + t you mean f(kx + t), not kf(x) + t? (The latter is not necessarily a valid pdf.) $\endgroup$
    – user3035
    Feb 12 '10 at 0:42
  • $\begingroup$ yeah, exactly (well, f((x-t)/k)) $\endgroup$
    – Neil
    Feb 13 '10 at 20:49

I guess I can't leave comments just yet so this will have to be in the form of an answer:

Assuming that the distributions have the same number of degrees of freedom ($n$), I think the answer will look something like (with some abuse of notation)

$\mathcal{H}(k_aA + t_a \mid k_bB + t_b) = \log(\frac{k_b}{k_a}) + \frac{n+1}{2} \left( E[ \log(1 + (\frac{Yk_a + t_a - t_b}{k_b})^2 \frac{1}{n}) ] - E[ \log (1 + \frac{Y^2}{n}) ]\right) $,

where the r.v. $Y$ has the Student's t distribution with $n$ degrees of freedom. If the number of degrees of freedom are assumed to be different between the two distributions you will get additional terms that correspond to the logarithm of the "$\Gamma$-parts" of the density functions and in the above formula the $n$'s will be changed accordingly. Expectation will still be w.r.t. the Student's t with $n$ deg. of freedom (or whatever degree that is associated with the r.v. $A$).


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.