# Find a way to apply the MLE on Fisher or Covariance matrix to make cross-correlations

I have 2 Fisher matrixes which represent information for the same variables (I mean columns/rows represent the same parameters in the 2 matrixes).

Now I would like to make the cross-correlations synthesis of these 2 matrixes by applying for each parameter the well known formula (coming from Maximum Likelihood Estimator method) :

$$\dfrac{1}{\sigma_{\hat{\tau}}^{2}}=\dfrac{1}{\sigma_1^2}+\dfrac{1}{\sigma_2^2}\quad(1)$$

$$\sigma_{\hat{\tau}}$$ represents the best estimator representing the combination of a sample1 ($$\sigma_1$$) and a sample2 ($$\sigma_2$$).

Now, I would like to do the same but for my 2 Fisher matrixes, i.e from a matricial point of view.

1. Firstly, for this, I tried to diagonalize in a simultaneous way each of these 2 Fisher matrix, i.e by finding a common eigenvectors basis for both. Then, I would add the 2 diagonal matrices and I have so a global diagonal Fisher matrix, and after come back in the space of start.

But this method gives the same constraints (by inversing the Fisher matrix) are the same than a classical synthesis between 2 matrices since :

$$V^{-1} A V = D_1$$

$$V^{-1} B V = D_2$$ ,

then what I wanted to do by summing the 2 diagonal matrices $$D_1$$ and $$D_2$$ is the same thing than doing $$A+B$$ :

$$A+B = V (D_1+ D_2) V^{-1}$$

Finally, I had to give up this track

1. Secondly, I tried to work directly in the space of Covariance matrix. I diagonalized "simultaneously" each of the 2 matrices. (Then, I have no more covariances terms).

and I build another covariance matrix by applying the MLE like putting on the diagonal :

$$\sigma_{\hat{\tau}}^{2}=\dfrac{1}{\dfrac{1}{\sigma_1^2}+\dfrac{1}{\sigma_2^2}}\quad(2)$$

Unfortunately, it doesn't increase the FoM (Figure of Merit, equal to $$\dfrac{1}{\sqrt{\text{det(block(2 parameters))}}}$$ ), I mean that constraints are not better : as a conclusion, I can't manage to do cross-correlations since I have no gain on constraints.

Moreover, with the first method 1) above, after the building of Fisher matrix and its inversion, I can marginalize over the nuisance parameters and re-invert to get a Fisher matrix where nuisance parameters estimations are encoded into it.

But I can't do the same for method 2). I can fix parameters (remove directly lines/columns) in Fisher matrix but this produces too high FoM (and so too small constraints) since I have less error parameters to estimate.

So, my question is about the research of a way to apply the MLE on Fisher (or maybe directly on Covariance matrix) to make cross-correlations between 2 given Fisher matrices (which represent actually 2 different probes into a cosmology context).

UPDATE 1: @fedja . Thanks a lot for your support and answer. I have a good and bad news. Firstly, it seems that you have formulated my issue on a pure mathematical point of view and when you finally find that :

$$Fisher_{\text{total}}=\left(\sum_j S_j^{-1}\right)\quad(1)$$

This makes sense with the classical combination between 2 Fisher matrix. Indeed, If I take $$S_j^{-1}$$ as a Fisher matrix (2 matrices actually by taking $$j=1,2$$), I find the classical synthesis of final covariance coming from summing all the Fisher matrices ( $$\left(\sum_j S_j^{-1}\right)$$) and finally inverse this sum of the Fisher matrices to get the final covariance matrix by doing :

$$S=\left(\sum_j S_j^{-1}\right)^{-1}\quad(2)$$

By the way, could you give me please the details to find the below expression $$(3)$$ which is appropriate when minimizing $$S=\sum_j U_jS_jU_j^*$$ :

$$U_j=(\sum_j S_j^{-1})^{-1}S_j^{-1}\quad(3)$$

The bad news is that I think I can't get further in cross-correlations between the 2 Fisher matrix from a mathematical point of view.

I have now to put more physics hypothesis. For example, here the formula in cosmology to compute the cross-correlated Fisher matrix for 2D (photometric and Weak lensing): an element $$(\alpha,\beta)$$ of this matrix is equal to :

By posting this message here, I thought that I could combine relatively easily combine the spectroscopic probe (3D) with 2D such that I would a final synthesis 2D+3D for my final Fisher matrix and so the fulll covariance matrix of the synthesis 2D+3D.

But unfortunately, I have to give up the hope of handling the case 2D+3D with only mathematics.

UPDATE 2: Just a last question : what do you think about pooled variance. I would like to find a relevant $$b$$ parameter in the expression :

$$s_{\text{total}} = bs_1 + (1-b)s_2 ; V(s_p)=b^2 V(s_1) + (1-b)^2 V(s_2)$$

where $$V(s_1)$$ and $$V(s_2)$$ represent the standard variance of my 2 samples, so the method called "Pooled variance" : here an example

Could you think it could help me by fixing this value to 0.5 (the first Fisher matrix brings same information as the second one) ? It would deserves another post on maths forum, doesn't it ?

• Why did I receive a downvote ? it seems to not be justified, I tried to do my best to clearly set the problematic. Jan 30, 2021 at 14:38
• The downvote seems a bit harsh, but I think your question could be a bit clearer. I don't work in statistics, so I can't tell which terms are standard, but you could try to clean up the presentation and set out more plainly what your question actually is. Jan 30, 2021 at 15:03
• @LeoMoos thanks for your remark I am going to try to be clearer. Jan 30, 2021 at 15:08
• I'm still having trouble understanding what you really want. It looks like some matrix optimization question but I cannot discern what it is without googling a lot of stuff first. I'm not even sure what your random variables are and what particular norm of error you want to minimize after combining the two observations. Maybe it is just my ignorance or stupidity, though :-) Feb 1, 2021 at 12:37
• @fedja The problematic is mainly based on Fisher formalism and combination between parameter errors given by the inverse of a Fisher matrix, I;e the covariance matrix. I suggest you to look at en.wikipedia.org/wiki/Fisher_information#Matrix_form . Don't hesitate to ask questions if necesssary. Feb 1, 2021 at 13:24

OK, let's turn the tables around. I'll describe in my language a physical setup, the mathematical formulation, and the answer that fit the three words I understand: "detector", "error", and "formula (1)" and you'll try to figure out whether they have anything to do with your problem :-)

Physical setup: You have an object whose true position $$x$$ is represented by a point in $$\mathbb R^n$$ and $$m$$ detectors that read it as $$x_j=x+y_j$$ ($$j=1,\dots,m$$) where $$y_j$$ is mean $$0$$ Gaussian noise with known correlation matrix $$S_j=Ey_jy_j^*$$. Noises for different detectors are independent and $$S_j$$ may be quite different. How to combine $$x_j$$ to get the least square error in every direction simultaneously?

Mathematical formulation: Put $$\hat x=\sum_j U_jx_j$$ where $$U_j$$ are arbitrary linear operators with $$\sum U_j=I$$ (to have an unbiased estimator). We want to minimize the covariance matrix $$S=\sum_j U_jS_jU_j^*$$ of $$\hat x-x$$ in the sense of positive definite quadratic forms.

Solution: (well-known, of course, but I'll provide the details if needed) Put $$U_j=(\sum_j S_j^{-1})^{-1}S_j^{-1}$$ (usual matrix inversion and multiplication) and get $$S=\left(\sum_j S_j^{-1}\right)^{-1}\tag 1$$ which is optimal in the sense that for any other choice of $$U_j$$, you'll get a quadratic form dominating this one.

Now let me know if this setup and the final result have anything to do with what you want :-)

Edit: The details requested by the OP.

If $$\hat x=\sum_j U_j x_j=\sum_j U_j(x+y_j)$$ and $$\sum_j U_j=I$$, then $$E(\hat x-x)(hat x-x)^*=\sum_{j,k} U_jEy_jy_k^*U_k^*=\sum_j U_jS_jS_j^*$$ because $$Ey_jy_k^*=0$$ if $$j\ne k$$ (independence) and $$Ey_jy_j^*=S_j$$ Hence the formula for the variance of the linear estimator.

Suppose now that we have some $$U_j$$ that satisfy $$\sum_j U_j=I$$. We want to show that $$\sum_j U_jS_jU_j^*\ge\left(\sum_j S_j^{-1}\right)^{-1}$$ in the sense of quadratic forms, i.e., that $$\sum_j \langle S_jU_j^*z,U_j^*z\rangle\ge \left\langle\left(\sum_j S_j^{-1}\right)^{-1}z,z\right\rangle$$ Note that $$\sum_jU_j^*=I^*=I$$, so $$z=\sum_jz_j$$ where $$z_j=U_j^*z$$. Now the desired inequality is just the reformulation of the convexity property of the mapping $$(W,z)\mapsto\langle W^{-1}z,z\rangle$$ where $$W$$ is running over the cone of positive semi-definite matrices and $$z$$ is running over $$\mathbb R^n$$. The choice of $$U_j$$ that I gave attains that low bound.

In general it looks like a very partial case of the Rao-Cramer lower bound for the variance of an unbiased estimator (which, I suspect, it is), so it shouldn't be surprising for you.

• Hi ! thanks for your answer, could you take a look please at my UPDATE 1, especially the details of demonstration of $$S=\sum_j U_jS_jU_j^*$$ and $$U_j=(\sum_j S_j^{-1})^{-1}S_j^{-1}\quad(3)$$ ? Best regards Feb 2, 2021 at 12:31
• Hi ! I hope you are in a good mood, if you could explain me the demonstration in my last comment, I would be grateful. Regards Feb 6, 2021 at 18:42
• @youpilat13 OK, I'll try to do it over the weekend :-) Feb 6, 2021 at 18:52
• Thanks for your help ! Feb 6, 2021 at 20:32
• @youpilat13 OK, I added the stuff you requested. Feel free to ask more questions but don't expect quick answers, just apologies for the delays :-) Feb 11, 2021 at 21:18