The relative entropy is frequently framed as the inverse temperature times the difference between the Helmholtz and variational/Bethe free energies, the latter of which is minimized as a proxy objective function in mean field theory and Bayesian estimation via the well-known expectation maximization algorithm.
Steve Huntsman
- 15.4k
- 7
- 75
- 130