Question

By polar decomposition, every continuous linear function $f \colon H \to K$ between Hilbert spaces can be written uniquely as $f = \widehat{f} \circ |f|$ for a positive operator $|f| \colon H \to H$ and a partial isometry $\widehat{f} \colon H \to K$ with $\ker(\widehat{f})=\ker(|f|)$. The binary operation $(f,g) \mapsto |g \circ f|$ on the set of positive operators is not associative. Is the binary operation $(f,g) \mapsto \widehat{g \circ f}$ on the set of partial isometries associative?

Answer 1

I have not fully checked this idea, but here goes. I prefer the notation $P(T)$ for the partial isometry occurring in the polar decomposition of $T$. I also got lost with three Hilbert spaces in the mix, so this answer is only for the case where $A,B,C$ all operate on the same fixed Hilbert space $H$.

In this notation, I believe the question is whether or not
$$ P(P(AB)C) = P(A P(BC)) $$ holds for all partial isometries $A,B,C$ on $H$.

I think it is easy to see that if $U$ is unitary, then for all $X$ we have $P(UX) = U P(X)$, and that if $T$ is a partial isometry, then $P(T) = T$. From this, it seems to follow that if $A$ is assumed unitary, and $B$ and $C$ are partial isometries, we have $P(P(AB)C) = P(AP(B)C) = P(ABC) = A P(BC)$ and $P(A P(BC)) = A P(P(BC)) = A P(BC)$ so the desired result holds.

If $A$ is not unitary, it still has a unitary dilation. This means there is another Hilbert space $K$ and operators $X: K \to H$ and $Y: K \to K$ with the property that the operator $A'$ on $H \oplus K$ given by the block operator matrix $A' = \begin{pmatrix} A & X \cr 0 & Y \end{pmatrix}$ is unitary. So consider the operators $B' = \begin{pmatrix} B & 0 \cr 0 & 0 \end{pmatrix}$ and $C' = \begin{pmatrix} C & 0 \cr 0 & 0 \end{pmatrix}$ on $H \oplus K$. The operators $B'$ and $C'$ are partial isometries on $H \oplus K$, so by the work above, $$ P(P(A'B')C') = P(A' P(B'C')). $$ Now calculate: $A'B' = \begin{pmatrix} AB & 0 \cr 0 & 0 \end{pmatrix}$ and so a moment's thought ought to show that $P(A'B') = \begin{pmatrix} P(AB) & 0 \cr 0 & 0 \end{pmatrix}$, making the left hand side of the above equal to $$ P(\begin{pmatrix} P(AB) C & 0 \cr 0 & 0 \end{pmatrix}) = \begin{pmatrix} P(P(AB) C) & 0 \cr 0 & 0 \end{pmatrix}. $$ The right hand side is almost the same, but with $P(A P(BC))$ in the upper left corner, and unless I made a ridiculous error, you have what you want. (There is a sightly more complicated unitary dilation theorem for operators between different spaces, so if there were no mistakes in this approach, maybe the same idea works in the general case too.)

Answer 2

Ah, (quite) some fiddling with Mathematica gave a counterexample. In the notation of anon's answer, take $$ A' = \begin{pmatrix} 1 & 1 & 1 & 1 \\ 1 & 1 & 1 & 0 \end{pmatrix}, \qquad B = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}, \qquad C = \begin{pmatrix} 0 & 1/\sqrt{2} \\ 1 & 0 \\ 0 & 1/\sqrt{2} \end{pmatrix}. $$ Then $A=P(A')$, $B$ and $C$ are partial isometries, but $P(P(CB)A) \neq P(CP(BA))$. Notice that $C$ is a partial isometry that is not an isometry, i.e. has nontrivial kernel.