For Hermitian matrices and operators, the most "natural" inner product is $f^H \cdot g$ or $\int f^* g\; dx$. A similar situation holds interpreting Fourier transforms as the inner product of functions with complex exponential functions. My question is, why is this the most "natural" choice? Is there something deeper to this choice (other than making $< f,f>$ a norm) related to some kind of duality of vector spaces?

Bi (or sesqui) linear forms are nicer if they're nondegenerate. But they can always be restricted to subspaces. So, they're even nicer if they're nondegenerate on all subspaces. For symmetric forms on ${\mathbb R}^n$, that forces definiteness (positive or negative). The usual bilinear form on ${\mathbb C}^n$ doesn't have this inheritednondegeneracy property, but the Hermitian one does. Anyway that's a practical issue, rather than a naturality statement. One way to decide it is natural is to embed $M_n({\mathbb C})$ into $M_{2n}({\mathbb R})$ by using the obvious $\mathbb R$basis of ${\mathbb C}^n$. This fills the $2n\times 2n$ matrix with lots of $2\times 2$ real matrices whose transposes correspond to complex conjugate. Transposes come up in dot products if you notice that $\langle v,w\rangle$ is the unique entry in the $1\times 1$ matrix $v^T w$. 


Not all inner products do in fact require conjugation. There are 8 elementary types of inner products on modules over the associative real division algebras: $\mathbb{R}$, $\mathbb{C}$ and $\mathbb{H}$. Over $\mathbb{R}$ and $\mathbb{C}$ you can consider bilinear inner products: symmetric and symplectic, whereas over $\mathbb{C}$ and $\mathbb{H}$ you can consider sesquilinear inner products: hermitian and skewhermitian. (Over $\mathbb{C}$ the distinction between hermitian and skewhermitian is just a factor of $i$, but over $\mathbb{H}$ it is really a different type of structure.) You can associate a classical group to each such type of inner product, corresponding to the transformations which leave the inner product invariant and having unit determinant:
Of these, only the hermitian and skewhermitian require conjugation, but the skewhermitian over $\mathbb{H}$ is not positive definite, hence does not give rise to a norm. The existence of the inner product says that as a representation of corresponding symmetry group $G$, (one of $\mathrm{SO}$, $\mathrm{Sp}$, $\mathrm{SU}$, $\mathrm{SO}^\ast$) the module $V$ is isomorphic either to the dual module $V^\ast$ (in the case of $\mathrm{SO}$, $\mathrm{Sp}$ and $\mathrm{SO}^\ast$) or to the conjugate dual module $\overline{V}^\ast$ (in the case of $\mathrm{SU}$). So one possible "high level" explanation (although it feels more like a rephrasing) of the fact that the hermitian inner product on a complex vector space requires conjugation is that the defining representation of the (special) unitary group is isomorphic to its conjugate dual and not to its dual. 

