Return to Answer

Here is a proof of $\det M\geq 0$ for $M\in \mathfrak{sp}(n)$ based on the lemma that every complex matrix is consimilar to a real matrix. Since the proof of that lemma does not rely on diagonalization in any obvious way, it should qualify the requirements of the OP.

Acknowledgment: In what follows I was helped by feedback I received at MSE.

By construction, the $2n\times 2n$ complex matrix $M\in \mathfrak{sp}(n)$ is skew-Hermitian and Hamiltonian, which means that it has the $n\times n$ block decomposition $$M=\begin{pmatrix} A&B\\ C&-A^T\end{pmatrix},\;\;\text{with}\;\;A=-A^\ast,\;\;B=B^T=-C^\ast=-\bar{C}.$$ Here $M^T$ denotes the transpose, $\bar{M}$ the complex conjugate, and $M^\ast$ the conjugate transpose.

By continuity of the determinant it is sufficient to consider $\det A\neq 0$. Then Schur's determinant identity gives

$$\det M=\det(-AA^T-ACA^{-1}B)=\det(A\bar{A}+A\bar{B}A^{-1}B)$$ $$\qquad=\det(A\bar{A})\det(1+\bar{A}^{-1}\bar{B}A^{-1}B)$$ $$\qquad=|\det A|^2\det(1+\bar{X}X),\;\;\text{with}\;\;X=A^{-1}B.$$

Now I apply the consimilarity lemma, to write $X=SR\bar{S}^{-1}$ with $R$ a real matrix. This gives $$\det M=|\det A|^2\det(1+\bar{S}R^2\bar{S}^{-1})=|\det A|^2\det(1+R^2)$$ $$\qquad=|\det A|^2\det(1+iR)\det(1-iR)$$ $$\quad=|\det A|^2|\det(1+iR)|^2\geq 0.$$

Here is a proof of $\det M\geq 0$ for $M\in \mathfrak{sp}(n)$ based on the lemma that every complex matrix is consimilar to a real matrix.

Acknowledgment: In what follows I was helped by feedback I received at MSE.

By construction, the $2n\times 2n$ complex matrix $M\in \mathfrak{sp}(n)$ is skew-Hermitian and Hamiltonian, which means that it has the $n\times n$ block decomposition $$M=\begin{pmatrix} A&B\\ C&-A^T\end{pmatrix},\;\;\text{with}\;\;A=-A^\ast,\;\;B=B^T=-C^\ast=-\bar{C}.$$ Here $M^T$ denotes the transpose, $\bar{M}$ the complex conjugate, and $M^\ast$ the conjugate transpose.

By continuity of the determinant it is sufficient to consider $\det A\neq 0$. Then Schur's determinant identity gives

$$\det M=\det(-AA^T-ACA^{-1}B)=\det(A\bar{A}+A\bar{B}A^{-1}B)$$ $$\qquad=\det(A\bar{A})\det(1+\bar{A}^{-1}\bar{B}A^{-1}B)$$ $$\qquad=|\det A|^2\det(1+\bar{X}X),\;\;\text{with}\;\;X=A^{-1}B.$$

Now I apply the consimilarity lemma, to write $X=SR\bar{S}^{-1}$ with $R$ a real matrix. This gives $$\det M=|\det A|^2\det(1+\bar{S}R^2\bar{S}^{-1})=|\det A|^2\det(1+R^2)$$ $$\qquad=|\det A|^2\det(1+iR)\det(1-iR)$$ $$\quad=|\det A|^2|\det(1+iR)|^2\geq 0.$$

My first attempt misunderstood the question, let me try again.

I need one lemma, which is that for any complex matrix $X$, with complex conjugate $\bar{X}$, the determinant $\det(1+\bar{X}X)\geq 0$. I asked at MSE, and was pointed to several proofs. I we allow for this lemma, the result of the OP follows simply:

By construction, the $2n\times 2n$ complex matrix $M$ is skew-Hermitian and Hamiltonian, which means that it has the $n\times n$ block decomposition $$M=\begin{pmatrix} A&B\\ C&-A^T\end{pmatrix},\;\;\text{with}\;\;A=-A^\ast,\;\;B=B^T=-C^\ast=-\bar{C}.$$ Here $M^T$ denotes the transpose, $\bar{M}$ the complex conjugate, and $M^\ast$ the conjugate transpose.

By continuity of the determinant it is sufficient to consider $\det A\neq 0$. Then Schur's determinant identity gives

$$\det M=\det(-AA^T-ACA^{-1}B)=\det(A\bar{A}+A\bar{B}A^{-1}B)$$ $$\qquad=\det(A\bar{A})\det(1+\bar{A}^{-1}\bar{B}A^{-1}B)$$ $$\qquad=|\det A|^2\det(1+\bar{X}X)\geq 0,\;\;\text{with}\;\;X=A^{-1}B.$$

Here is a proof of $\det M\geq 0$ for $M\in \mathfrak{sp}(n)$ based on the lemma that every complex matrix is consimilar to a real matrix. Since the proof of that lemma does not rely on diagonalization in any obvious way, it should qualify the requirements of the OP.

Acknowledgment: In what follows I was helped by feedback I received at MSE.

By construction, the $2n\times 2n$ complex matrix $M\in \mathfrak{sp}(n)$ is skew-Hermitian and Hamiltonian, which means that it has the $n\times n$ block decomposition $$M=\begin{pmatrix} A&B\\ C&-A^T\end{pmatrix},\;\;\text{with}\;\;A=-A^\ast,\;\;B=B^T=-C^\ast=-\bar{C}.$$ Here $M^T$ denotes the transpose, $\bar{M}$ the complex conjugate, and $M^\ast$ the conjugate transpose.

By continuity of the determinant it is sufficient to consider $\det A\neq 0$. Then Schur's determinant identity gives

$$\det M=\det(-AA^T-ACA^{-1}B)=\det(A\bar{A}+A\bar{B}A^{-1}B)$$ $$\qquad=\det(A\bar{A})\det(1+\bar{A}^{-1}\bar{B}A^{-1}B)$$ $$\qquad=|\det A|^2\det(1+\bar{X}X),\;\;\text{with}\;\;X=A^{-1}B.$$

Now I apply the consimilarity lemma, to write $X=SR\bar{S}^{-1}$ with $R$ a real matrix. This gives $$\det M=|\det A|^2\det(1+\bar{S}R^2\bar{S}^{-1})=|\det A|^2\det(1+R^2)$$ $$\qquad=|\det A|^2\det(1+iR)\det(1-iR)$$ $$\quad=|\det A|^2|\det(1+iR)|^2\geq 0.$$

My first attempt mistook the question, let me try again.

By construction, the $2n\times 2n$ complex matrix $M$ is skew-Hermitian and Hamiltonian, which means that it has the $n\times n$ block decomposition $$M=\begin{pmatrix} A&B\\ C&-A^T\end{pmatrix},\;\;\text{with}\;\;A=-A^\ast,\;\;B=B^T=-C^\ast=-\bar{C}.$$ Here $M^T$ denotes the transpose, $\bar{M}$ the complex conjugate, and $M^\ast$ the conjugate transpose.

Let me assume $\det A\neq 0$. Then Schur's determinant identity gives

$$\det M=\det(-AA^T-ACA^{-1}B)=\det(A\bar{A}+A\bar{B}A^{-1}B)$$ $$\qquad=\det(A\bar{A})\det(1+\bar{A}^{-1}\bar{B}A^{-1}B)$$ $$\qquad=|\det A|^2\det(1+\bar{X}X),\;\;\text{with}\;\;X=A^{-1}B.$$

It remains to prove that for any complex matrix $X$, the determinant $\det(1+\bar{X}X)\geq 0$, which requires some further steps. (I have asked at MSE.)

My first attempt misunderstood the question, let me try again.

I need one lemma, which is that for any complex matrix $X$, with complex conjugate $\bar{X}$, the determinant $\det(1+\bar{X}X)\geq 0$. I asked at MSE, and was pointed to several proofs. I we allow for this lemma, the result of the OP follows simply:

By construction, the $2n\times 2n$ complex matrix $M$ is skew-Hermitian and Hamiltonian, which means that it has the $n\times n$ block decomposition $$M=\begin{pmatrix} A&B\\ C&-A^T\end{pmatrix},\;\;\text{with}\;\;A=-A^\ast,\;\;B=B^T=-C^\ast=-\bar{C}.$$ Here $M^T$ denotes the transpose, $\bar{M}$ the complex conjugate, and $M^\ast$ the conjugate transpose.

By continuity of the determinant it is sufficient to consider $\det A\neq 0$. Then Schur's determinant identity gives

$$\det M=\det(-AA^T-ACA^{-1}B)=\det(A\bar{A}+A\bar{B}A^{-1}B)$$ $$\qquad=\det(A\bar{A})\det(1+\bar{A}^{-1}\bar{B}A^{-1}B)$$ $$\qquad=|\det A|^2\det(1+\bar{X}X)\geq 0,\;\;\text{with}\;\;X=A^{-1}B.$$