0
$\begingroup$

I came across the following note in a paper I'm reading and don't understand how it was derived.

$\max_{\alpha_\ell}\sum_\ell^L\beta_\ell\log\alpha_\ell$ such that $\sum_\ell^L\alpha_\ell=1$ and $\alpha_\ell\geq 0 \forall\alpha_\ell$ (that is, $\alpha$ is a probability distribution) is maximized by:

$\alpha_\ell*=\frac{\beta_\ell}{\sum_{\ell'}^L\beta_{\ell'}}$

I've tried the classic take a partial derivative and set equal to 0, but cannot figure out where the $\ell'$ comes from and keep getting 0=0, which is not helpful. What am I missing?

Thanks!

Improve this question
$\endgroup$

1 Answer 1

Reset to default
1
$\begingroup$

When you take partial derivatives, substitute $\alpha_1=1-\sum_{\ell\ne 1}\alpha_\ell$ to get $$0=\beta_{\ell}/\alpha_{\ell}-\frac{\beta_1}{\left(1-\sum_{\ell\ne 1}\alpha_\ell\right)}$$ for each $\ell$. This means that $\beta_\ell/\alpha_\ell$ does not depend on $\ell$, which implies the solution you want.

Improve this answer
$\endgroup$
2
  • $\begingroup$ Thanks for your answer! Is the derivative not $\sum_\ell\frac{\beta_\ell}{\alpha_\ell}$? In which case I'm not sure how you got rid of the sum and got the $-\frac{\beta_1}{\alpha_1}$. I'm sure you're right though--it's been a while since I've done any of this! $\endgroup$
    – nachtm
    Commented Jun 30, 2017 at 14:12
  • $\begingroup$ Wait never mind, you did the substitution before deriving, got it. I still don't know where the sum went though! $\endgroup$
    – nachtm
    Commented Jun 30, 2017 at 15:03

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .