A question on vectors in $\mathbb{R}^4$

Question

Let $M$ be an invertible $4 \times 4$ real matrix, and let $\mathbf{u}, \mathbf{v} \in \mathbb{R}^4$. Consider the matrix $$\displaystyle \mathcal{M} = \mathcal{M}(\mathbf{u}, \mathbf{v}) = [\mathbf{u} | M \mathbf{u} | \mathbf{v} | M \mathbf{v} ].$$ Here the notation is that the columns of $\mathcal{M}$ are $\mathbf{u} , M \mathbf{u} , \mathbf{v} , M \mathbf{v}$ respectively.

For generic $M$ and for generic pairs $\mathbf{u}, \mathbf{v}$, $\mathcal{M}$ will be invertible and so the only vector orthogonal to all four columns will be the zero vector. If $\mathbf{u}$ and $\mathbf{v}$ are proportional then $\mathcal{M}$ will have rank at most two, and there is no way of determining a unique direction which is orthogonal to all of the columns. However, there exist pairs $\mathbf{u}, \mathbf{v}$ such that $\mathcal{M}$ has rank three, and hence $\ker \mathcal{M}^T$ is 1-dimensional and there should be an algebraic formula which gives a generator in terms of $\mathbf{u}, \mathbf{v}$ and $M$. Our $M$ will always be chosen so that generically $\mathcal{M}$ is invertible and there exist $\mathbf{u}, \mathbf{v}$ such that $\mathcal{M}$ has rank three.

The motivation here is a generalization of the cross-product in $\mathbb{R}^3$. There the goal is to find an algebraic expression which gives the generator of the kernel of $[\mathbf{u} | \mathbf{v}]^T$, namely the cross product $\mathbf{u} \times \mathbf{v}$.

One way to do this is to note that the condition that $\ker \mathcal{M}^T$ is non-trivial is equivalent to the condition that $\det(\mathcal{M}) = 0$. Expanding, we find that

$$\displaystyle \det(\mathcal{M}) = Q_M(\mathbf{u}, \mathbf{v})$$

which is a bi-quadratic form in $\mathbf{u}, \mathbf{v}$. Fixing $\mathbf{u}$, we then obtain a quadratic form $\mathcal{Q}_{M, \mathbf{u}}(\mathbf{v})$ in $\mathbf{v}$, which has an obvious solution $\mathbf{v} = \mathbf{u}$. Using the general theory of quadratic forms one can then parametrize all solutions to the equation $\mathcal{Q}_{M, \mathbf{u}}(\mathbf{v}) = 0$. From here one can then find the kernel by taking a Laplace expansion, just like in the 3-dimensional case with the cross product.

Is there a more "linear" method of describing the set, and perhaps in a way which is more symmetric in $\mathbf{u}, \mathbf{v}$?

Answer 1

If ker ${\cal M}^{T} $ is one-dimensional, there are two possibilities:

Case 1: $(\mathbf{u},\mathbf{v},M\mathbf{u})$ are linearly dependent; then $M\mathbf{v} $ is linearly independent of the other three, and we can construct the vector $\mathbf{w} $ orthogonal to all four vectors via $\mathbf{w}_{i} = \epsilon_{ijkl} \mathbf{u}_{j} \mathbf{v}_{k} (M\mathbf{v} )_{l} $.

Case 2: $(\mathbf{u},\mathbf{v},M\mathbf{u})$ are linearly independent; then $M\mathbf{v} $ is linearly dependent on the other three, and we can construct the vector $\mathbf{w} $ orthogonal to all four vectors via $\mathbf{w}_{i} = \epsilon_{ijkl} \mathbf{u}_{j} \mathbf{v}_{k} (M\mathbf{u} )_{l} $, and we can also add arbitrary multiples of $\epsilon_{ijkl} \mathbf{u}_{j} \mathbf{v}_{k} (M\mathbf{v} )_{l} $, as long as the result doesn't cancel.

We can furthermore consolidate the two cases into the expression $$ \mathbf{w}_{i} = \epsilon_{ijkl} \mathbf{u}_{j} \mathbf{v}_{k} [(M\mathbf{u} )_{l} + (M\mathbf{v} )_{l} +\mbox{sgn}(\epsilon_{mnop} \mathbf{u}_{n} \mathbf{v}_{o} (M\mathbf{u} )_{p} \epsilon_{mqrs} \mathbf{u}_{q} \mathbf{v}_{r} (M\mathbf{v} )_{s})(M\mathbf{v} )_{l}] $$ because in either case, it reduces to the form specific to that case already given. Here, the last term in the square brackets is a kludge to take into account the loophole pointed out by Willie Wong in comments, that the $M\mathbf{u} $ and $M\mathbf{v} $ components orthogonal to both $\mathbf{u} $ and $\mathbf{v} $ may cancel; this last term eliminates such a cancellation. Maybe one can make this less ugly and completely symmetric in $\mathbf{u} $, $\mathbf{v} $ again.