# What is an intuitive view of adjoints? (version 2: functional analysis)

After realising that I don't have an intuitive understanding of adjoint functors, I then realised that I don't have an intuitive understanding of adjoint linear transformations!

Again, I can use 'em, compute 'em, and convolute 'em, but I have no real intuition as to what is going on. My best attempts (when teaching them) were:

1. The adjoint allows us to shift stuff from one side of the inner product to the other, thus, in a fashion, moving it out of the way while we do something and then moving it back again.
2. Nice behaviour with respect to the adjoint (say, normal or unitary) translates into nice behaviour with respect to the inner product.

But neither of those is really about what the adjoint is, just what it does.

-

Just to add to Yemon's answer, in the case of inner product spaces I think it may be helpful to understand the case of operators of finite rank. So let's start with the shockingly simple case where $\eta$ is a single vector in the inner product space $H$. I like to identify this with the linear map $t\mapsto t\eta$ from $\mathbb{C}$ to $H$, whose adjoint is the map $\xi\mapsto\langle\xi,\eta\rangle$ – in other words, just the functional associated with $\xi$. Going the other way (assuming $H$ is complete), the adjoint of a linear functional “is”, of course, just the vector provided to us by the Riesz representation theorem. To physicists, who write the inner product backwards (with good reason, btw) we are mere looking at the bra and ket version of the same vector, so $\langle\eta|$ and $|\eta\rangle$ will be each others' adjoints.

Next, you can tackle rank one operators between Hilbert spaces, which will be of the form $\xi\mapsto\langle\eta,\xi\rangle\zeta$ for fixed vectors $\eta$ and $\zeta$. And once you have a fairly good idea how that works, it's a short way to adding a finite number of these to get any finite rank operator.

Admittedly, from here to infinite rank is something of a stretch, but I think it might help anyhow.

-
I like this idea: I'm only after an intuitive idea so the passage to infinite dimensions can be "hand-waved away". I'm going to think about what self-adjoint, skew-adjoint, and normal look like in this description (unitary may be a little tricker!) – Loop Space Nov 23 '09 at 15:22
A comment is too short to explain why I'm accepting this answer so I'm posting a "community wiki" answer to explain. – Loop Space Nov 24 '09 at 9:06

It seems like you're thinking about adjoints with respect to an inner product, but I find it more natural to think of adjoints as a way to get a map $B^{\ast} \to A^{\ast}$ from a map $A \to B$ and then to think of inner products as distinguished choices of isomorphisms $B \simeq B^{\ast}, A \simeq A^{\ast}$.

That's what I tell people, anyway. Secretly, the way I view adjoints is as the generalization of the following operation: if $M$ is the biadjacency matrix of a directed bipartite graph, then $M^{\ast}$ is the biadjacency matrix of the opposite graph, with all edges reversed. In other words, if you think of linear operators from a Markov chain perspective as encoding transition probabilities between states, then the adjoint linear operator is about running those transitions backwards.

-

I've accepted Harald Hanche-Olsen's answer to this question. However, I feel that it needs a little expansion to explain exactly why. The short reason is that his answer triggered a connection in my brain that meant that a few idiosyncrasies of Hilbert spaces suddenly became clear. I'd like to explain them. I should add that I fully expected this question to be one of those where there wasn't a "right" answer.

The key point was that his answer made me think about the rank one case where everything is much simpler. As he said, a vector in $H$ "is" a linear map $\mathbb{C} \to H$, $t \mapsto t \eta$. Similarly, a functional on $H$ is a linear map $H \to \mathbb{C}$. Riesz rep says that such a functional is of the form $\xi \mapsto \langle \eta, \xi \rangle$. Put them together and you find that rank one operators $H \to H$ are of the form $\xi \mapsto \langle \eta, \xi\rangle \zeta$. Or in "bra-ket" notation, $|\xi\rangle \mapsto |\zeta\rangle \langle \eta | \xi \rangle$. So the operator is $|\zeta \rangle \langle \eta|$.

Next is to think about adjoints. What's the adjoint of this operator? Simple: swap everything around. So $|\zeta \rangle \langle \eta|^\star = |\eta \rangle \langle \zeta|$. From this we can deduce that "self-adjoint" means "$\eta = \zeta$", "skew-adjoint" means "$\eta = i\zeta$", whilst "normal" is a little more complicated.

However, this doesn't feel very insightful. The "click" was to think about $|\zeta \rangle \langle \eta|$ and $|\eta \rangle \langle \zeta|$ and try to work out what the swap operation was doing. The block, for me, on why this is purely an algebraic thing and not at all geometric is that the representation of functionals as "inner-product-with-vector" is one of those "it just works" theorems, with little insight as to why it works (yes, yes, I know why, I just don't know why). But if one removes Riesz Representation altogether then one is simply working in a Banach space and adjoints don't exist. So I needed to remove the conclusion of Riesz Rep, but leave enough of it to still ensure that I know I'm in a Hilbert space.

The key step in Riesz Rep is the complementary subspace property: that each subspace has a complement (turns out that this characterises Hilbert spaces, as proved by Lindenstrauss and Tzafriri in 1971). This is much more intuitive for me: it's geometric.

So when we think of a rank one linear transformation on $H$, we have two obvious subspaces: the image and the kernel. The image has dimension $1$, the kernel has codimension $1$. The complementary subspace property then implies that each of these has a complement, the complement of the image being of codimension $1$ and of the kernel being dimension $1$. So up to scalar, these specify another linear transformation. And that linear transformation is the adjoint. Then everything else falls into place.

So the key, for me, is that the fact that $\ker A^\star = (\operatorname{Im} A)^\perp$ and vice versa is not a happy consequence of adjoints but rather the almost defining property.

That "almost" rescues me a little from having missed this before. The phrase "up to scalar" that I highlighted above is important. To get the true adjoint one has to do a little more work to show that there is an operator that satisfies this "almost defining" property and that the map $T \mapsto T^\star$ is "nice" - it's easier to go the other way around and start with the adjoint and work backwards.

Or one could do something a little different. Instead of saying "this works up to scalars, can we fix that?" we should say "this works up to scalars, so let's ignore scalars" and, being Good Geometers and Topologists we know how to do that! Work with projective spaces, and more generally with Grassmannians.

Concentrating again on rank one operators, we have that their images lie in $\mathbb{P}H$ and their kernels in $\mathbb{P}^{\infty - 1}H$ (closed codimension 1 subspaces), so up to scalars, a rank one operator is a point in $\mathbb{P}H \times \mathbb{P}^{\infty -1}H$. The crucial fact that the closest point property gives is that $\mathbb{P}H \cong \mathbb{P}^{\infty - 1}H$ (Riesz Rep is then the trivial observation that $\mathbb{P}^{\infty - 1}X \cong \mathbb{P}X$ for any normed vector space $X$). So really a rank one operator is a point in $\mathbb{P}H \times \mathbb{P}H$. Adjunction is then the obvious $\mathbb{Z}/2$ action on this.

It's not difficult to extend this to any finite rank operator, as Harald says. The question is to make the leap to infinite. At this point, Harald says:

Admittedly, from here to infinite rank is something of a stretch, but I think it might help anyhow.

which just goes to show that he doesn't know his own brilliance. The amazing thing is that there is no stretch! In a Hilbert space we have the approximation property which says that every continuous linear operator is the weak limit of a sequence of finite rank operators. Since adjunction is a weak property - in that you know that you have the adjoint if a load of evaluations say that you do - if I have a weakly convergent sequence of operators then the adjoints weakly converge to the adjoint! And once I have the adjoint, it is the adjoint no matter how bizarrely I found it.

In conclusion, it was Harald Hanche-Olsen's answer that triggered this in my brain so he gets the "accepted" tag. It may not be the "right" answer, but it was the answer that taught me a heck of a lot about something that I thought I knew a lot about already.

(PS Obviously, this answer is community wiki so voting for it gains me no rep and merely says that you agree with or like what I've said. However, if you do like this answer enough to vote for it, you should vote for Harald's answer as well since that sparked all of this one.)

-
I'm brilliant now? Thanks. And thanks for reminding me of how the complemntary subspace property characterized Hilbert spaces. I added a reference to the Lindenstrauss–Tzafriri paper proving it. – Harald Hanche-Olsen Nov 24 '09 at 14:11
Thanks! I wasn't sure enough of the names to warrant a steeple-chase trying to find the reference. – Loop Space Nov 24 '09 at 14:44
@HaraldHanche-Olsen & Andrew Stacey Thanks for briliant explanations and clarifications. I would like to point out that ["every continuous linear operator is the weak limit of a sequence of finite rank operators"]---> this is also very useful in algebra. Look at my answer here ("math.stackexchange.com/questions/1696798/">In which sense is composition a tensor product) – Duchamp Gérard H. E. Mar 17 at 5:26

For me the adjoint has everything to do with graphs of operators. Consider the graph of the operator A, i.e. the set (x,Ax) in the product space. This product space has a natural inner-product and we can look at the orthogonal complement of the graph in this product space. It is easy to see that the elements (y,z) in the orthogonal complement are exactly those elements with y=-A^*z. That for me gives a nice interpretation of the adjoint: it is the operator that up to a minus sign and switching coordinates has the graph that is orthogonal to the graph of A.

-
Very nice! I like having a geometric picture like this. – Loop Space Dec 8 '09 at 12:30

It doesn't answer your question, but just to throw out some tired and half-baked thoughts that might form the nucleus for someone else's more sensible answers:

For me, the adjoint of an operator is just what happens when you dualize a continuous map between Banach spaces. (Of course one doesn't need to work in this setting, but it's the one where I've spent most time.) There are then some standard little factlets, most of which can probably be extracted out of the right bits of Rudin's book or other good texts: an operator has dense range (in the norm topology of the target space) if and only if its adjoint is injective; an operator is injective with closed range if and only if its adjoint is injective.

But as you say, these are more about what the adjoint does, not what it is. I guess that the best I can say is that the existence of the adjoint of an operator is just a witness to the fact that duality of Banach spaces is (contravariantly) functorial. You'd get similar notions of adjoints whenever you have a closed category and fix an object C and look at the functor Hom(_,C).

As to what's going on in Hilbert space, which seems to be really what you were asking, there I'm even less able to express what they mean, as opposed to what they do and all the nice properties they have.

-
I really want adjoint in the Hilbert space setting (though one can generalise this), not dual maps. I understand dual maps, I think, but I want to understand how T^* and T are related. – Loop Space Nov 23 '09 at 14:26

Like everybody else, the first definition of an adjoint operator was given in terms of an inner product space. Only much later, I decided that this is the wrong way to think of an adjoint operator; I now find it much easier to think of it as follows:

First, remember that the dual space is the space of (continuous) scalar-valued linear functions on the original vector space.

If A: X -> Y is a linear map, then its adjoint is a map A*: Y* -> X*. Moreover, the definition of the adjoint map is quite natural and obvious: it is composition of each element in Y* with A.

The observation that the composition with A maps level sets to level sets leads naturally to the geometric interpretation that A* describes the inverse images of hyperplanes under A.

There is also the nice duality between how A (or A*) acts on subspaces and the other acts on quotient spaces.

I find that in general the inner product creates a lot of confusion (at least in me), because it creates an "unnatural" isomorphism between a vector space and its dual. Most of linear algebra is GL(n)-invariant (or equivariant) and is more easily understood in that context without an inner product. Also, by doing this, it is clearer to me what the effect of the inner product is.

-
This is sort of backward to what I was looking for. I know about dual maps and, to me, the important part of the adjoint is nothing to do with duality. The inner product isn't something to be wished away but is part of the initial structure. It's just that sometimes it's unclear exactly what the right way to think of that initial structure is. – Loop Space Nov 24 '09 at 14:46
OK. What the inner product does for you is to identify the dual space with the original space, where dual vectors should be viewed as normal vectors to a family of parallel hyperplanes. Then the adjoint map, viewed as a map on the original vector space should be viewed as mapping normal vectors to normal vectors (instead of hyperplanes to hyperplanes). – Deane Yang Nov 24 '09 at 16:34