Skip to main content
3 of 3
deleted 6 characters in body

It is in theory possible to use free probability to describe the limit eigenvalue distribution (as the dimension tends to $+\infty$) of the anticommutator $AB+BA$ when $A$ and $B$ are independent random matrices which are unitarily invariant (which is the case for the description used in the OP). The same is true actually for any self-adjoint polynomial in $A$, $B$.

The drawback of this method it that the limit distribution will be obtained in an indirect way. If one does this for the uniform measure on $[0,1]$ as asked in the OP, it may be hard to decide from this approch whether the limit distribution is supported on $\mathbf{R}^+$ or not.

However, such computations are possible in simple cases. Propositon 6.11 from Fevrier, Maxime; Mastnak, Mitja; Nica, Alexandru; Szpojankowski, Kamil, Using Boolean cumulants to study multiplication and anti-commutators of free random variables says the following: if $E$ and $F$ are subspaces of dimension $n/2$ which are chosen independently according to the $O(n)$-invariant measure on the Grassmann manifold, and $A$ and $B$ are the corresponding orthogonal projections, then as $n \to \infty$ the eigenvalue distribution of $AB+BA$ is described by an explicit density which is supported on $[-1/4,2]$ (see also Figure 2 in the same paper). In particular, the probability that $AB+BA$ is positive semidefinite tends to $0$ as $n \to \infty$. For this toy model, this gives a rigorous proof of the fact that the anticommutator of positive matrices is typically not positive. (This is probably overkill, ane can presumably analyze everything in terms of principal angles between the subspaces $E$ and $F$.)