Timeline for Multiplicative gradient descent?
Current License: CC BY-SA 3.0
4 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 15, 2014 at 4:04 | comment | added | Cristóbal Guzmán | Precisely. And most importantly, in all these 'experts' type of algorithms you are implicitly searching for probability distributions $\sum_i w_i=1$, $w_i\geq 0$, which is a simplex setup and thus you can use entropy as distance generating function. That's where the formulae come from. | |
| Sep 15, 2014 at 2:17 | vote | accept | thinkbear | ||
| Sep 15, 2014 at 2:17 | comment | added | thinkbear | Thanks! I guess the update $w_{t+1}=w_t[-\lambda_t\nabla f(w_t)]$ update, which is a "direct" multiplicative extension of additive gradient descent, is not meaningful anyway, it needs to be exponentiated first to make the variables remain positive. This falls into the MD framework. | |
| Sep 14, 2014 at 23:05 | history | answered | Cristóbal Guzmán | CC BY-SA 3.0 |