Timeline for The derivative of the Cholesky factor
Current License: CC BY-SA 3.0
32 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Jan 4 at 16:09 | comment | added | Dadeslam | I am a bit late in the comment, but I try. I am working in a similar setting, and if I understood correctly the Jacobian involving the vectorizations, since it has $I+K$ in it, is not invertible. On the other hand, the one with the half-vectorization is invertible. Can I recover the Jacobian of the vectorization in terms of the one with the half-vectorization? | |
| Jun 9, 2021 at 17:42 | comment | added | stollenm | @pete The previous attempt yielded the same result. | |
| Jun 9, 2021 at 11:28 | comment | added | pete | @stollem Well obviously I'm a lot easier to convince when the result is correct ;) Did you find the mistake in your earlier attempts? | |
| Jun 9, 2021 at 9:39 | comment | added | stollenm | For those interested in a more citeable resource. The result in this answer has been derived in Lemma 1 of doi.org/10.1016/0304-4076(89)90059-6. | |
| Jun 9, 2021 at 9:39 | comment | added | stollenm | @pete man are you difficult to convince :) . It does check out numerically as well, obviously. | |
| Jun 8, 2021 at 21:47 | comment | added | pete | @stollenm That sounds a little more promising. Let me know when you've tested it and I'll have a look. | |
| Jun 7, 2021 at 15:06 | comment | added | stollenm | @pete In fact you can write $Q = \frac{1}{2} (DL)^\top$ using Lemma 4.4 in epubs.siam.org/doi/abs/10.1137/0601049. | |
| Apr 16, 2021 at 7:19 | comment | added | pete | @stollenm I take that to mean no, you haven't checked. Get back to me when you have. | |
| Apr 16, 2021 at 7:10 | comment | added | pete | @stollenm It "should" work? Have you tried it? Does it give the same answer for $\frac{d\operatorname{vech}A}{d\operatorname{vech}P}$? | |
| Apr 16, 2021 at 6:13 | comment | added | stollenm | @pete If $D$ is the duplication matrix and $L$ is an elimination matrix (which is not unique, see math.stackexchange.com/a/4081768/185047) then we can just take the Moore-Penrose inverse of $D$ as $L$, $L=(DD')^{-1}D'$, in which case it should work as I said. | |
| Apr 16, 2021 at 5:43 | comment | added | pete | @stollenm I don't think so? The first $DL$ is transposed. | |
| Aug 14, 2020 at 13:59 | comment | added | Dorian | @pete thanks a lot, yes I used that and compared to the numerical derivative and it worked fine! however, I have a bit more advanced question that I posted here, would you mind having a look at that as well? (I odn't want to extend that comment thread even more..) | |
| Aug 14, 2020 at 9:11 | comment | added | pete | Hi @Dorian, it's been a while since I've looked at this but unless I've missed something $dL^T = (dL)^T$ as you suggest. Note that $\frac{\partial L}{\partial A}$ is a fourth-order tensor (and not a particularly nice one) which I think is why the chain rule isn't used directly here. | |
| Aug 12, 2020 at 14:56 | comment | added | Dorian | @pete and when using the chain differentiation with $A = f(\theta)$, the derivative, $\frac{\partial L}{\partial \theta} = L \Phi(L^{-1}\frac{\partial A}{\partial \theta} L^{-T})$, and not following the chain-rule: $\frac{\partial L}{\partial \theta} = \frac{\partial L}{\partial A} \cdot \frac{\partial A}{\partial \theta}$? | |
| Aug 12, 2020 at 14:51 | comment | added | Dorian | @pete what happens when I want to take the derivative of $A^T$? when I'm using this paper, the perturbation becomes $dL^T = (dL)^T = (L \Phi(L^{-1}dA L^{-T}))^T = \Phi(L^{-1}dA L^{-T})^T L^T$, is that correct? | |
| Feb 7, 2020 at 6:47 | comment | added | Steven Pav | I think intended it to mean the derivative with respect to the vech of $X$, but there is an oddity here in that $X$ need not be symmetric. | |
| Feb 6, 2020 at 0:44 | comment | added | JDoe2 | Hi, I was just wondering if anyone could clarify, what does the notation $\text{vech}_{\Delta}\left(X\right)$ mean above? Is this the vech of the cholesky factor? | |
| Sep 2, 2014 at 4:57 | comment | added | pete | Using that same paper, we can write $Q=\frac12(DL)^\top DL$ since $D^\top D = (L(I+K)L^\top)^{-1}$. | |
| Sep 2, 2014 at 2:36 | comment | added | pete | For a lower triangular matrix $A$, $\operatorname{vec}A = L^\top\operatorname{vech}A$. Ref: The Elimination Matrix: Some Lemmas and Applications. | |
| Sep 1, 2014 at 2:42 | comment | added | pete | I can generalise that result to give me $\frac{d\operatorname{vech}A}{d\operatorname{vech}P} = L(I\otimes A)Q(A^{-1}\otimes A^{-1})D$, where $Q$ is the diagonal matrix that gives us $Q\operatorname{vec}M = \operatorname{vec}(\Phi(M))$, and $D$ is the usual duplication matrix for symmetric matrices. This is more direct, and inverts $A$ rather than the larger matrix, but needs us to introduce the function $\Phi$ and matrix $Q$. | |
| Aug 31, 2014 at 8:25 | comment | added | pete | Theorem A.1 of Simo Särkkä's Bayesian Filtering and Smoothing gives a result for the scalar derivative $\frac{\partial A}{\partial\theta} = A\Phi\left(A^{-1}\frac{\partial P}{\partial\theta}{}A^{-\top}\right)$, where $P=AA^{\top}$, and $\Phi_{ij}(M) = M_{ij}$ where $i > j$; $\tfrac12M_{ij}$ where $i=j$; and $0$ where $i < j$. Using this I get the same thing as using your result above. | |
| Aug 31, 2014 at 6:41 | comment | added | pete | Columns of $L$ are either basis vectors or zero vectors. The submatrix of $L^{\top}L$ corresponding to non-zero elements of $M$ is the identity and the rest is zeros. | |
| Aug 30, 2014 at 22:10 | comment | added | Steven Pav | I suspect you are right, but have to convince myself again that $L^{\top}$ is the 'half' duplication matrix... | |
| Aug 30, 2014 at 6:54 | comment | added | pete | I prefer your result, it's a little more direct. I wonder if you should change $D$ to $L^{\top}$ though, it took me a bit of thinking before I realised that $D$ isn't the full duplication matrix. | |
| Aug 30, 2014 at 5:44 | comment | added | Steven Pav | good find, @pete ! I believe you are right, but wonder if the F&O derivative could be somehow used for Cholesky decomposition--perhaps by considering the matrix 'square root' function. | |
| Aug 30, 2014 at 3:43 | comment | added | pete | I think the paper is here, but it appears to talk about the eigendecomposition, not the Cholesky. | |
| Jun 24, 2014 at 19:13 | comment | added | Steven Pav | I am lead to believe there is a form of this in a paper by Fujikoshi and Okamoto in J. Japan. Stat. Soc., but I can find no more than the TOC for this journal: jss.gr.jp/ja/journal/jjss1976.html . | |
| May 27, 2014 at 21:18 | vote | accept | Steven Pav | ||
| May 27, 2014 at 21:18 | history | edited | Steven Pav | CC BY-SA 3.0 |
added 1144 characters in body
|
| May 25, 2014 at 4:22 | history | edited | Steven Pav | CC BY-SA 3.0 |
modulo the truth...
|
| May 20, 2014 at 19:02 | vote | accept | Steven Pav | ||
| May 20, 2014 at 22:31 | |||||
| May 20, 2014 at 19:02 | history | answered | Steven Pav | CC BY-SA 3.0 |