2
$\begingroup$

I'm reading this paper about training energy-based models (EBMs) and don't understand the parameters that we are training for? The part that is relevant to the question is in pages 1-4. Here is the overview.

EBM (Equation 1 in paper) refers to a model where the probability density has the form: $$p_\theta(\pmb{x})=\frac{\exp(-E_\theta(\pmb{x}))}{Z_\theta}$$ where $\pmb{x},\theta\in \mathbb{R}^n$, $E_\theta$ is an energy function, $Z_\theta=\int \exp(-E_\theta(\pmb{x}))d\pmb{x}$ is a normalizing constant. Ising model is an example of an EBM.

In equation 3, the gradient of log-probability of an EBM is given which is just \begin{equation}\triangledown_\theta \log p_\theta(\pmb{x})=-\triangledown_\theta E_\theta(\pmb{x})-\triangledown_\theta Z_\theta \label{eq:1}\end{equation}

Then the second term in the equation above ($\triangledown_\theta Z_\theta$) is approximated by $-\triangledown_\theta E_\theta(\tilde{\pmb{x}})$ in equation 4. Here $\tilde{\pmb{x}}$ is a random sample from $p_\theta(\pmb{x})$ i.e. $\tilde{\pmb{x}} \sim p_\theta(\pmb{x})$. Page 4 then talks about how to get $\tilde{\pmb{x}}$ using MCMC (Markov chains Monte Carlo) methods which I understand.

So I understand that in equation 3, the term $-\triangledown_\theta E_\theta(\pmb{x})$ is exactly computable and the term $\tilde{\pmb{x}}$ is approximated using MCMC methods.

  1. But I don't understand what we are training for?
  2. In each step of the training process, we get an estimate for $\triangledown_\theta \log p_\theta(\pmb{x})$ using equation 3 but somehow that is used to learn some parameter which I don't seem to get?
  3. In MLE one estimates the parameter $\theta$ such that the likelihood of the observed data conditional on $\theta$ is maximized. So are we doing a similar thing here and training for $\theta$?
  4. If the answer to 3) is yes, then how does $\triangledown_\theta \log p_\theta(\pmb{x})$ help in updating the parameter $\theta$?
Improve this question
$\endgroup$
7
  • $\begingroup$ Given approximations to the gradient of log likelihood you can get local maxima of the log likelihood as a function of $\theta$ - start anywhere, and then do gradient accent. At no point do you actually know what the log likelihood is, but you know you're getting better (stochastically). $\endgroup$
    – Dror Speiser
    Commented Dec 15, 2022 at 8:32
  • $\begingroup$ @DrorSpeiser I see, that makes sense, thanks! $\endgroup$
    – voila
    Commented Dec 15, 2022 at 16:29
  • $\begingroup$ @DrorSpeiser I understand that log likelihood is maximized using gradient ascent. In a vanilla Maximum Likelihood estimation, there is a notion of observed data. Do we observe any data here? Thanks. $\endgroup$
    – voila
    Commented Dec 17, 2022 at 0:52
  • $\begingroup$ Yes, the idea is to maximize $\mathbb{E}_{x\sim p_{data}} \log(p_\theta(x)) $. So what we're really doing with the gradient ascent is taking the mean over gradients at all $x$ in our data $\endgroup$
    – Dror Speiser
    Commented Dec 17, 2022 at 9:47
  • 1
    $\begingroup$ Just in case you haven't seen them, I'll quickly add that Song has a few nice YouTube videos from earlier this year about this research and paper, and there's also Ermon's IAS talk from two years ago. $\endgroup$
    – Dror Speiser
    Commented Dec 17, 2022 at 22:37

0

Reset to default

You must log in to answer this question.