MathOverflow will be down for maintenance for approximately 3 hours, starting Monday evening (06/24/2013) at approximately 9:00 PM Eastern time (UTC-4).

2 deleted 35 characters in body

It is not always the case that the composite of two operators is nice. In general, the composite of $S$ and $T$ is defined by $$D(ST)={\xi\in D(T) | T\xi\in D(S)}.$$ There is no reason to expect this space to be dense.

I agree with @Yemon. You want the affiliated operators. Here's an interesting parallel for finite measure spaces that I heard in a mini-course from Ozawa. Just as $$L^\infty(X,\mu)\subset L^1(X,\mu)\subset L^2(X,\mu)$$ which is a subset of all measurable functions, if $(M,tr)$ is a finite von Neumann algebra, $$M=L^\infty(M,tr)\subset L^1(M,tr)\subset L^2(M,tr)\subset \eta(M),$$ the affiliated operators. The inclusion for $L^2(M)$ into $\eta(M)$ is given by $$\xi\mapsto (L_\xi\colon m\mapsto \xi m),$$ (be careful - you need to take the closure of $L_\xi$. Clearly $L_\xi$ commutes with right multiplication by $U(M)$) and the image of $L^2(M)$ is the set of all $T\in \eta(M)$ such that if $$|T|=\int t dE(t)$$ is the usual spectral measure, then $$\int t^2 d tr(E(t)) <\infty.$$

About your comment in response to @Yemon: Indeed the trace works. If you have a positive function, use indicator functons to show the trace is semi-finite and faithful (see Takesaki I). Normal is clear. If you have a measurable function $f$ on $(X,\mu)$, construct a multiplication operator by letting $$D(M_f)={\xi\in L^2(X,\mu)| f\xi \in L^2(X,\mu)}.$$ If $\mu$ is finite, and $f$ is a measurable, real valued, a.e. finite function, then $M_f$ is self-adjoint, and the spectrum is the essential range of $f$.

Reed and Simon have a good introduction to unbounded operators, in particular the spectral theorem.

Somehow the brackets aren't showing up for the sets above, but I hope it's legible.

1

It is not always the case that the composite of two operators is nice. In general, the composite of $S$ and $T$ is defined by $$D(ST)={\xi\in D(T) | T\xi\in D(S)}.$$ There is no reason to expect this space to be dense.

I agree with @Yemon. You want the affiliated operators. Here's an interesting parallel for finite measure spaces that I heard in a mini-course from Ozawa. Just as $$L^\infty(X,\mu)\subset L^1(X,\mu)\subset L^2(X,\mu)$$ which is a subset of all measurable functions, if $(M,tr)$ is a finite von Neumann algebra, $$M=L^\infty(M,tr)\subset L^1(M,tr)\subset L^2(M,tr)\subset \eta(M),$$ the affiliated operators. The inclusion for $L^2(M)$ into $\eta(M)$ is given by $$\xi\mapsto (L_\xi\colon m\mapsto \xi m),$$ (be careful - you need to take the closure of $L_\xi$. Clearly $L_\xi$ commutes with right multiplication by $U(M)$) and the image of $L^2(M)$ is the set of all $T\in \eta(M)$ such that if $$|T|=\int t dE(t)$$ is the usual spectral measure, then $$\int t^2 d tr(E(t)) <\infty.$$

About your comment in response to @Yemon: Indeed the trace works. If you have a positive function, use indicator functons to show the trace is semi-finite and faithful (see Takesaki I). Normal is clear. If you have a measurable function $f$ on $(X,\mu)$, construct a multiplication operator by letting $$D(M_f)={\xi\in L^2(X,\mu)| f\xi \in L^2(X,\mu)}.$$ If $\mu$ is finite, and $f$ is a measurable, real valued, a.e. finite function, then $M_f$ is self-adjoint, and the spectrum is the essential range of $f$.

Reed and Simon have a good introduction to unbounded operators, in particular the spectral theorem.

Somehow the brackets aren't showing up for the sets above, but I hope it's legible.