For Hermitian matrices and operators, the most "natural" inner product is $f^H \cdot g$ or $\int f^* g\; dx$. A similar situation holds interpreting Fourier transforms as the inner product of functions with complex exponential functions. My question is, why is this the most "natural" choice? Is there something deeper to this choice (other than making $< f,f>$ a norm) related to some kind of duality of vector spaces?

5$\begingroup$ It has something to do with the fact that the Hom functor is contravariant in the first variable. Baez's HigherDimensional Algebra II: 2Hilbert spaces (arxiv.org/abs/qalg/9609018) might be a good place to start reading, but I'm not sure it's a complete explanation. $\endgroup$ – Qiaochu Yuan Jul 26 '11 at 0:28

2$\begingroup$ I always wondered about this myself... $\endgroup$ – Igor Rivin Jul 26 '11 at 0:32

2$\begingroup$ This kind of inner products makes orthogonality a symmetric relation. If you have a bilinear form B on some vector space V, call two vectors v and w orthogonal if B(v,w) = 0. One would like this to be a symmetric relation: if B(v,w) = 0 then B(w,v) = 0. One can show (see Grove, Classical Groups and Geometric Algebra, Prop. 2.7) that having orthogonality w.r.t. B being symmetric is equivalent to B being a symmetric or alternating bilinear form (which lead to orthogonal and symplectic groups, respectively). [continued...] $\endgroup$ – KConrad Jul 26 '11 at 4:26

5$\begingroup$ On a complex vector space, with sesquilinear forms B in place of bilinear forms, one can ask when the orthogonality relation B(v,w) = 0 is symmetric in v and w. If B is nondegenerate in a suitable sense, then orthgonality w.r.t. B is a symmetric relation if and only if B is a scalar multiple of a Hermitian form. This, to me, explains in an interesting why why symmetric bilinear forms, alternating bilinear forms, and Hermitian sesquilinear forms are special: they are essentially the interesting ways of making a geometry in which perpendicularity is a symmetric relationship. $\endgroup$ – KConrad Jul 26 '11 at 4:28
Bi (or sesqui) linear forms are nicer if they're nondegenerate. But they can always be restricted to subspaces. So, they're even nicer if they're nondegenerate on all subspaces. For symmetric forms on ${\mathbb R}^n$, that forces definiteness (positive or negative).
The usual bilinear form on ${\mathbb C}^n$ doesn't have this inheritednondegeneracy property, but the Hermitian one does.
Anyway that's a practical issue, rather than a naturality statement. One way to decide it is natural is to embed $M_n({\mathbb C})$ into $M_{2n}({\mathbb R})$ by using the obvious $\mathbb R$basis of ${\mathbb C}^n$. This fills the $2n\times 2n$ matrix with lots of $2\times 2$ real matrices whose transposes correspond to complex conjugate. Transposes come up in dot products if you notice that $\langle v,w\rangle$ is the unique entry in the $1\times 1$ matrix $v^T w$.

3$\begingroup$ Looking up the definition of a sesquilinear form led to its Wikipedia article, which gives a nice naturality argument for both bilinear and sesquilinear forms. $\endgroup$ – Victor Liu Jul 26 '11 at 3:49

1$\begingroup$ Maybe I am wrong, but I think nondegeneracy does not imply definiteness. A direct sum of a positive and a negative definite bilinear form is still nondegenerate (using the def from wikipedia) and it is not definite (if both forms were nontrivial). $\endgroup$ – HenrikRüping Jul 26 '11 at 15:35

6$\begingroup$ Indefiniteness of the space implies that some subspace will be degenerate. $\endgroup$ – Allen Knutson Jul 26 '11 at 23:12
Not all inner products do in fact require conjugation. There are 8 elementary types of inner products on modules over the associative real division algebras: $\mathbb{R}$, $\mathbb{C}$ and $\mathbb{H}$. Over $\mathbb{R}$ and $\mathbb{C}$ you can consider bilinear inner products: symmetric and symplectic, whereas over $\mathbb{C}$ and $\mathbb{H}$ you can consider sesquilinear inner products: hermitian and skewhermitian. (Over $\mathbb{C}$ the distinction between hermitian and skewhermitian is just a factor of $i$, but over $\mathbb{H}$ it is really a different type of structure.) You can associate a classical group to each such type of inner product, corresponding to the transformations which leave the inner product invariant and having unit determinant:
 $\mathrm{SO}$ for symmetric bilinear over $\mathbb{R}$ and $\mathbb{C}$
 $\mathrm{Sp}$ for skewsymmetric bilinear over $\mathbb{R}$ and $\mathbb{C}$ and also hermitian over $\mathbb{H}$
 $\mathrm{SU}$ for hermitian (and skewhermitian) over $\mathbb{C}$
 $\mathrm{SO}^\ast$ for skewhermitian over $\mathbb{H}$
Of these, only the hermitian and skewhermitian require conjugation, but the skewhermitian over $\mathbb{H}$ is not positive definite, hence does not give rise to a norm.
The existence of the inner product says that as a representation of corresponding symmetry group $G$, (one of $\mathrm{SO}$, $\mathrm{Sp}$, $\mathrm{SU}$, $\mathrm{SO}^\ast$) the module $V$ is isomorphic either to the dual module $V^\ast$ (in the case of $\mathrm{SO}$, $\mathrm{Sp}$ and $\mathrm{SO}^\ast$) or to the conjugate dual module $\overline{V}^\ast$ (in the case of $\mathrm{SU}$).
So one possible "high level" explanation (although it feels more like a rephrasing) of the fact that the hermitian inner product on a complex vector space requires conjugation is that the defining representation of the (special) unitary group is isomorphic to its conjugate dual and not to its dual.

2$\begingroup$ And by the way, all 8 classes of inner products appear as invariant inner products on the representations of Clifford algebras. This is explained in the book Spinors and Calibrations by Reese Harvey. $\endgroup$ – José FigueroaO'Farrill Jul 26 '11 at 1:06

$\begingroup$ I've never seen the notation $SO^*$. And it is a classical group?! $\endgroup$ – Mariano SuárezÁlvarez Jul 26 '11 at 1:15

$\begingroup$ Your answer is a bit above my level of understanding (I come from an engineering background, but dabble in mathematical physics). It seems that your enumeration of various inner products is motivated by symmetries they possess (e.g. hermiticity), and perhaps my question is then: why are these symmetries the most natural? Maybe there is just no answer to this. I come about this question looking at inner products for quasinormal modes (in the context of open systems in optics or gravitation) which extend selfadjointness to open systems by defining a "better" inner product. $\endgroup$ – Victor Liu Jul 26 '11 at 3:58

$\begingroup$ @Mariano: I'm not sure if $\mathrm{SO}^\ast$ is standard notation, but it is used in some books on Lie groups, e.g., Rossmann's. It is the defined as the group of $\mathbb{H}$linear transformations in $\mathbb{H}^n$, thought of as a right $\mathbb{H}$module, which preserves the skewhermitian inner product given by $z^\ast j w$ for $z,w \in \mathbb{H}^n$. $\endgroup$ – José FigueroaO'Farrill Jul 26 '11 at 9:59