The cofactor map $A\mapsto\widehat A$ is polynomial homogeneous of degree $n-1$ over ${\bf M}_n({\mathbb R})$. It can be polarized into an $(n-1)$-linear symmetric map. When $n=3$, this provides a bilinear map $(A,B)\mapsto\widehat{A,B}$, defined by $$\widehat{A,B}=\frac12\left(\widehat{A+B}-\widehat A-\widehat B\right).$$

Let me focus on the subspace of symmetric matrices. If $A$ is symmetric (resp. positive definite), then so is $\widehat A$. If $A,B\in{\bf Sym}_3$, then obviously $\widehat{A,B}$ is symmetric. More interesting and a little less obvious is the fact that if $A,B\in{\bf SPD}_3$, then $\widehat{A,B}$ is still positive definite. This is a consequence of the fact that the determinant is a hyperbolic polynomial over ${\bf Sym}_3$, with future cone ${\bf SPD}_3$ ; actually the result extends to positive symmetric matrices of arbitrary size.

Now let me recall the geometric mean of positive definite matrices $A,B$ : $$X\sharp Y=Y^{\frac12}\left(Y^{-\frac12}XY^{-\frac12}\right)^{\frac12}Y^{\frac12}.$$ My question is about comparing two symmetric positive definite matrices:

Is it true that whenever $A,B\in{\bf SPD}_3$, we have $$\widehat A\sharp\widehat B\prec\widehat{A,B},\qquad(\dagger)$$ in the sense of the order between quadratic forms  ?

Remark that both sides are homogeneous of degree $1$ with respect to either argument. I have a positive answer in the following subcases:

1. $A=I_3$, then it reduces to the arithmetic-geometric inequality,
2. $A\vec e_1=0$ (which is a limit case when $A$ is semi-definite), the calculation being more involved.
3. $B=A$, trivial because both sides equal $\widehat A$.

The inequality $(\dagger)$ can be seen as a variant of Garding's Inequality for hyperbolic polynomials. If $P:{\mathbb R}^N\to{\mathbb R}$ is homogeneous of degree $d$, hyperbolic with forward cone $\Gamma$, then the associated $d$-linear form $\phi$ satisfies $$\phi(a_1\ldots,a_d)\ge P(a_1)^{\frac1d}\cdots P(a_d)^{\frac1d},\qquad\forall a_1,\ldots,a_d\in\Gamma.$$ Here $d=2$, ${\mathbb R}^N\sim{\bf Sym}_3$, the cofactor map stands for $P$, and the inequality between numbers is replaced by Loewner's order between symmtric matrices.

The cofactor map $A\mapsto\widehat A$ is polynomial homogeneous of degree $n-1$ over $\mathbf M_n({\mathbb R})$. It can be polarized into an $(n-1)$-linear symmetric map. When $n=3$, this provides a bilinear map $(A,B)\mapsto\widehat{A,B}$, defined by $$\widehat{A,B}=\frac12\left(\widehat{A+B}-\widehat A-\widehat B\right).$$

Let me focus on the subspace of symmetric matrices. If $A$ is symmetric (resp. positive definite), then so is $\widehat A$. If $A,B\in\mathbf{Sym}_3$, then obviously $\widehat{A,B}$ is symmetric. More interesting and a little less obvious is the fact that if $A,B\in\mathbf{SPD}_3$, then $\widehat{A,B}$ is still positive definite. This is a consequence of the fact that the determinant is a hyperbolic polynomial over $\mathbf{Sym}_3$, with future cone $\mathbf{SPD}_3$; actually the result extends to positive symmetric matrices of arbitrary size.

Now let me recall the geometric mean of positive definite matrices $A$, $B$: $$X\mathbin\sharp Y=Y^{\frac12}\left(Y^{-\frac12}XY^{-\frac12}\right)^{\frac12}Y^{\frac12}.$$ My question is about comparing two symmetric positive definite matrices:

Is it true that whenever $A,B\in\mathbf{SPD}_3$, we have $$\widehat A\mathbin\sharp\widehat B\prec\widehat{A,B},\tag{$\dagger$}\label{dagger}$$ in the sense of the order between quadratic forms?

Remark that both sides are homogeneous of degree $1$ with respect to either argument. I have a positive answer in the following subcases:

1. $A=I_3$, then it reduces to the arithmetic-geometric inequality,
2. $A\vec e_1=0$ (which is a limit case when $A$ is semi-definite), the calculation being more involved.
3. $B=A$, trivial because both sides equal $\widehat A$.

The inequality \eqref{dagger} can be seen as a variant of Gårding's Inequality for hyperbolic polynomials. If $P:{\mathbb R}^N\to{\mathbb R}$ is homogeneous of degree $d$, hyperbolic with forward cone $\Gamma$, then the associated $d$-linear form $\phi$ satisfies $$\phi(a_1,\dotsc,a_d)\ge P(a_1)^{\frac1d}\dotsb P(a_d)^{\frac1d},\qquad\forall a_1,\dotsc,a_d\in\Gamma.$$ Here $d=2$, ${\mathbb R}^N\sim\mathbf{Sym}_3$, the cofactor map stands for $P$, and the inequality between numbers is replaced by Loewner's order between symmetric matrices.