Timeline for Continuity of entropies, replica trick and Hausdorff moment problem
Current License: CC BY-SA 4.0
12 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jul 30 at 14:37 | comment | added | Dabed | For any m you choose $c_m^{*}=\text{argmax}I^{(m)}(c)\ge c^{*}_{log}=\text{argmax}I(c)$ so for all of them, I would try again here on MO or on MSE too surely someone can be more helpful than me. | |
| Jul 30 at 14:19 | comment | added | nervxxx | hm.. where in your argument do you use the knowledge of all $m$? | |
| Jul 30 at 14:07 | comment | added | Dabed | Yeah didn't see it treated as multidimensional from the question but doing the unidimensional optimization $c^{*}=\text{argmin} (y-xc)^2=x^{-1}y$ isn't so different from doing least squares $C^{* }=\text{argmin}||Y-XC||^2=(X^tX)^{-1}X^tY$ for example, so in doing $\partial_c\mathbb{E}(p^{m-1}(x|c))=0,\partial_c\mathbb{E}(\log p(x|c))=0$, the $\partial_c$ should be changed to $\nabla_c$, not sure if m equal a specific value should matter for both to give the same value for $c^{*}$. Maybe reformulating the problem in a new question on MO (or MSE too) could give you more and better help. | |
| Jul 30 at 8:06 | comment | added | nervxxx | @Dabed actually, just to understand what you are doing: note the parameter $c = (c_1, c_2, \cdots,c_n)$ can be multi-dimensional (in the particular example it is 1-dim). what does the inequality you showed on $c_*$ mean? Element-wise? | |
| Jul 30 at 4:21 | comment | added | Dabed | Now I think at least one direction should be true as because of Jensen's inequality you should have $$c*=\text{argmax}I^{(m)}(c)=\text{argmax}\mathbb{E}(p^{m-1}(x|c))=\text{argmax}\log\mathbb{E}(p^{m-1}(x|c))\ge_\text{Jensen}\text{argmax} \mathbb{E}(\log p^{m-1}(x|c))=\text{argmax}(m-1)\mathbb{E}(\log p(x|c))=\text{argmax}\mathbb{E}(\log p(x|c))=\text{argmax}I(c)$$, about m=2 not sure I follow why it should be sufficient. | |
| Jul 30 at 3:29 | comment | added | nervxxx | @Dabed thanks! your argument would seem to suggest only the $m=2$ information is sufficient.. | |
| Jul 30 at 3:15 | comment | added | Dabed | In maximum likelihood estimation is used that $\text{argmax} L(\theta;y)=\text{argmax}\log L(\theta)$ by monotonicity of log so maybe the same can be done under the expectation $c*=\text{argmax}I^{(m)}(c)=\text{argmax}\mathbb{E}(p^{m-1}(x|c))=_{?}\text{argmax} \mathbb{E}(\log p^{m-1}(x|c))=\text{argmax}(m-1)\mathbb{E}(\log p(x|c))=\text{argmax}\mathbb{E}(\log p(x|c))=\text{argmax}I(c)$, not sure what is needed to make $\text{argmax} \mathbb{E}(p^{m-1}(x|c))=_{?}\text{argmax} \mathbb{E}(\log p^{m-1}(x|c))$ if even possible although. | |
| Jul 29 at 13:13 | history | edited | nervxxx | CC BY-SA 4.0 |
added 67 characters in body
|
| Jul 29 at 10:09 | history | edited | nervxxx | CC BY-SA 4.0 |
added 18 characters in body
|
| Jul 29 at 4:36 | history | edited | nervxxx | CC BY-SA 4.0 |
deleted 24 characters in body
|
| Jul 29 at 3:15 | history | edited | nervxxx | CC BY-SA 4.0 |
added 22 characters in body
|
| Jul 29 at 2:07 | history | asked | nervxxx | CC BY-SA 4.0 |