## Extremal points of the unit ball of M_n(R) …

The unit ball of ${\bf M}_n(\mathbb R)$ is a compact convex subset. As such, it is (Krein-Milman theorem) the convex envelop of its extremal points. So far, so good; but the unit ball depends of the choice of a norm. There are a few natural choices:

• the Schur-Frobenius (Hilbert-Schmidt) norm $\|X\|_F=\sqrt{{\rm Tr}X^TX}$. This is the standard Euclidian norm over ${\bf M}_n(\mathbb R)\sim\mathbb R^{n^2}$. The extremal points form the unit sphere.
• the operator norm $\|X\|_2$ associated with the Euclidian norm over $\mathbb R^n$. The extremal points form the orthogonal group ${\bf O}_n(\mathbb R)$. See a related question.
• the operator norm $\|X\|_1$ associated with the $\ell_1$-norm over $\mathbb R^n$. The extremal points are the sign-permutation matrices. There are only $2^nn!$ of them.
• the numerical radius $r(X)=\sup|y^*Xy|$, where the supremum is taken over all unit complex vectors (real vectors are not enough). It is the smallest radius of a disk $D(0;r)$ containing the numerical range (Hausdorffian) of $X$. So, this my question:

Let $B_{\rm nr}$ be the unit ball when we endow the $n\times n$-matrices with the norm $r$. What are the extremal points of $B_{\rm nr}$ ?

Edit. To answer Geoff's comment. If $A$ is a normal matrix, then $r(A)=\rho(A)$ (the numerical range is the convex envelop of the spectrum in this case). But if $A$ is not normal, one usually have (not always) $r(A)>\rho(A)$. Thus not all matrices with spectral radius equal to $1$ belong to $B_{\rm nr}$. And even if $A$ is normal and $\rho(A)=1$, it is not extremal unless all eigenvalues have unit modulus.

Re-edit. After thinking to the case $n=2$, I have the opinion that considering the unit ball for $r$ in ${\bf M}_n(\mathbb R)$ is unnecessarily complicated. It would be more reasonable to work in ${\bf M}_n(\mathbb C)$ instead.

-
+1: another nice question Denis! – S. Sra Mar 2 2012 at 15:46
To answer the last questian: the extrem points of $B_{nr}$ are not included in $O_n(R)$ (resp. do not contain $O_n(R)$). The reason: otherwise it would imply that $\|A\|\leq 1$ for all $A \in B_{nr}$ (resp. $\begin{pmatrix} 0 & 1\\1 &0\end{pmatrix} = 1/2 \begin{pmatrix} 0 & 2\\0 &0\end{pmatrix} + 1/2 \begin{pmatrix} 0 & 0\\2 &0\end{pmatrix}$). – Mikael de la Salle Mar 2 2012 at 17:50
@Mikael. You're right (actually I found the same identity soon after editing), except that $\pm I_n$ is an extremal point of $B_{\rm nr}$. – Denis Serre Mar 3 2012 at 8:08

After working a little bit, I solved the $2\times 2$ case. Surprisingly, ${\rm ext}(B_{\rm nr})$ is not closed. It consists of (recall that if $n=2$, the numerical range is an ellipse $E_A$ together with its interior; the ellipse may degenerate to a segment or a point)

• homotheties $zI_2$ with $|z|=1$,
• matrices $A$ whose ellipse $E_A$ is tangent interiorly to $\bf S^1$ at two points $w,z$ such that $w+z\ne0$,
• nilpotent matrices $N$ with $\|N\|_F=2$. This is the case where $E_N=\bf S^1$,
• normal matrices with distinct eigenvalues $w,z$ such that $w+z\ne0$ and $|w|=|z|=1$.

It is the condition $w+z\ne0$ which prevents ${\rm ext}(B_{\rm nr})$ from being closed.

The second class above may be seen as the general one, the other ones being limits of it.

See the details in the draft.

-