Some integrals with respect to a Gaussian measure on a Hilbert space Assume that $(H,\langle\cdot,\cdot\rangle)$ is a real separable Hilbert space equipped with a Gaussian measure $\mu$ with a mean $m$ and a covariance operator $C$. Let $x\in H$ be a fixed vector. What are the values of the following integrals:
$$\int_H\frac{|\langle x,h\rangle|}{\|h\|}\mu(\mbox{d}h)$$
and
$$\int_H\frac{|\langle x,h\rangle|^2}{\|h\|^2}\mu(\mbox{d}h),$$
can we express these integrals in terms of just $x$ and an inner product or norm?
 A: First integral: no, this expression is not polynomial in $x$ due to the absolute value sign in the numerator, so it cannot be written as an inner product or norm of $x$.  It certainly can be expressed in terms of an inner product (for instance, in the form you already have), and I argue below that any finite number of terms in its series expansion (in $C$) should be obtainable in closed form.
Second integral: this expression is a seminorm.  Whether it is a norm depends on $C$.  Evaluating it is also not easy to my knowledge, but should be easier than the first integral.
I cannot speak to the infinite dimensional case.
Your first integrand is not differentiable in $h$.  I'd suggest integrating separately over the half-spaces separated by the plane $\langle x, h\rangle=0$.  In each half space, I'd perform the Taylor expansion around the $m$, then carry out the Gaussian integrals over the dimensions orthogonal to $x$ via Isserlis's theorem.  The final integration parallel to $x$ should leave you with a series of expressions in closed form in terms of incomplete Gamma functions.
My inner engineer notes that if $m$ is large compared to the root-eigenvalues of $C$, then
$$
\int_H\frac{\left|\langle x,h\rangle\right|}{\|h\|}\mu(\mbox{d}h)\approx \mathrm{sgn}(\langle x,m\rangle) \left\langle x, \int_H\frac{h}{\|h\|}\mu(\mbox{d}h) \right\rangle
\\
= \mathrm{sgn}(\langle x,m\rangle) \left\langle x, \int_H\frac{m+\xi}{\|m+\xi\|}\nu(\mbox{d}\xi) \right\rangle,
$$
where $\nu$ is a Gaussian measure over $H$ with mean $0$ and covariance operator $C$.
$$
= \mathrm{sgn}(\langle x,m\rangle) \left\langle x, \int_H(m+\xi)\left(\langle m,m\rangle + 2 \mbox{Re}\langle m,\xi\rangle + \langle \xi, \xi\rangle\right)^{-1/2}\nu(\mbox{d}\xi) \right\rangle
\\
\approx \frac{\mathrm{sgn}(\langle x,m\rangle)}{\|m\|} \left\langle x, \int_H(m+\xi)\left(1 - \frac{\langle m,\xi\rangle}{\|m\|^2} - \frac12\frac{\langle \xi, \xi\rangle}{\|m\|^2}+\frac32\frac{\langle m,\xi\rangle^2}{\|m\|^4}\right)\nu(\mbox{d}\xi) \right\rangle
\\
= \frac{\mathrm{sgn}(\langle x,m\rangle)}{\|m\|} \left[\langle x, m\rangle\left(1 - \frac{\mbox{Tr}C}{2\|m\|^2} + \frac{3\langle m, C m\rangle}{2\|m\|^4}\right)
- \frac{\langle x,Cm\rangle}{\|m\|^2}\right]
\\
= \left|\langle x, \hat m\rangle\right|\left(1 - \frac12\mbox{Tr}\hat C + \frac32\langle \hat m, \hat C \hat m\rangle\right)
- \mathrm{sgn}(\langle x,\hat m\rangle) \langle x,\hat C\hat m\rangle,
\\
\text{where }\hat C = C/\|m\|^2, \hat m = m/\|m\|.
$$
For your second integral,
$$
\int_H \frac{\left|\langle x,h\rangle\right|^2}{\|h\|^2}\mu(\mathrm{d}h) = \left\langle x\right| \left(\int_H \frac{\left|h\right\rangle\left\langle h\right|}{\|h\|^2}\mu(\mathrm{d}h)\right) \left|x\right\rangle
$$
The integral is quadratic in $x$.  If $C$ is positive definite, then the operator in parentheses is a norm, since the integrand is a (positive semidefinite) orthogonal projector onto the subspace parallel to $h$, and the integration measure is nonzero over a full-dimensional region of $H$.  In general, it looks like it is a seminorm, e.g. in the case where the kernel of $C$ includes vectors orthogonal to $m$.
The matrix elements of this operator could, in general, be evaluated by Taylor expanding around $m$, then using Isserlis' theorem to write the resulting Gaussian integrals over monomials in terms of traces of products of $C$ and $m$.
So I think the answer to your question is mostly "yes, in certain limiting cases".
