Conditions on $\beta$ under which the trace pairing restricted to $\mathfrak{so}(V,\beta)$ is positive (negative) definite

Question

Let $V$ be a finite dimensional vector space over $ \mathbb{R}$. Let \begin{equation} \left\langle\:,\:\right\rangle:\mbox{End}(V)\otimes\mbox{End}(V)\rightarrow \mathbb{R}\end{equation} denote the pairing given by $\left\langle A,B\right\rangle = \mbox{Tr}(AB)$. We know that this pairing is $\mbox{GL}(V)$-invariant, symmetric, non-degenerate, but not definite (i,e. neither positive definite nor negative definite).

Supposed that $\beta:V\otimes V \rightarrow \mathbb{R}$ is a non-degenerate symmetric pairing. Recall $\mathfrak{so}(V,\beta)\subset \mathfrak{gl}(V)$ is defined by $$\mathfrak{so}(V,\beta)=\left\{A\in\mathfrak{gl}(V)\:|\: \forall v,w\in V \: \beta(Av,w)+\beta(v,Aw)=0\right\}.$$ We know that $\mathfrak{so}(V,\beta)$ is a Lie subalgebra of $\mathfrak{gl}(V)$.

The question: Establish conditions on $\beta$ under which the restriction of the trace pairing (i,e. $\left\langle\:,\:\right\rangle$ restricted to $\mathfrak{so}(V,\beta)$) is positive (also conditions to be negative) definite.

My attempt: We know that given $\beta:V\times V\rightarrow \mathbb{R}$ a non-degenerate symmetric form, for all $A\in \mbox{End}(V)$ there exist a unique $B\in \mbox{End}(V)$ which satisfies $\beta(Av,w)=\beta(v,Bw)$ for all $v,w\in V$.

The unique $B$ as above will be denoted by $A^{\dagger}$ and called the transpose of $A$ (with respect to $\beta$).

The assignment $A\rightarrow A^{\dagger}$ defines a linear map $(\cdot)^{\dagger}\mbox{End}(V)\rightarrow \mbox{End}(V)$ called transposition (with respect to $\beta$).

Therefore, we can consider $$\mathfrak{so}(V,\beta)=\left\{A\in\mathfrak{gl}(V)\:|\: A=-A^{\dagger} \: (\mbox{transpose with respect to }\beta)\right\}.$$ In this sense, we have

1. Restriction the trace pairing is positive definite if $\mbox{Tr}(A^{\dagger}A)<$ for all $A\in \mathfrak{so}(V,\beta)$ with $A\neq 0$.
2. Restriction the trace pairing is negative definite if $\mbox{Tr}(A^{\dagger}A)>$ for all $A\in \mathfrak{so}(V,\beta)$ with $A\neq 0$.

My question: Conditions 1 and 2 depend on $\beta$. Does the condition 1 and 2 answer the question? Can I characterize $A^{\dagger}$ in terms of $\beta$ of a better way? Are there more specific conditions on $\beta$ to answer the question?

Answer 1

Yes to all three questions. By diagonalisation of forms, WLOG, $\beta$ is diagonal with $\pm 1$ on the main diagonal. Now let us just compute (and you can get an explicit formula for your transposition as well).

If $\beta$ is positive or negative definite, $so(V,\beta)$ is the set of skew-symmetric matrices. The trace form is negative definite as easily observed.

If $\dim V =2$ and $\beta$ is indefinite, $so(V,\beta)$ is one-dimensional. The trace form is positive definite as easily observed.

If $\dim V >2$ and $\beta$ is indefinite, you look at a subalgebra of $so(V,\beta)$, of matrices that are zero outside a diagonal $3\times 3$-minor, which you choose so that $\beta$ has both $+1$ and $-1$ in the corresponding positions. You can easily write two matrices $A$ and $B$ in this subalgebra such that $Tr(A^2)>0<Tr(B^2)$. Thus, the trace form is indefinite.