Let $H$ be a reproducing kernel Hilbert space of functions $f:E\to F$ with kernel $k$. A point in $E$ may be embedded into $H$ via the canonical embedding $x\mapsto k(x,\cdot)$. Similarly, a random variable that takes values in $E$ defines a canonical $H$-valued random variable via the mapping $X\mapsto k(X, \cdot)$. Let $X_H := k(X, \cdot)$. Then $X_H$ "encodes" much of the information about $X$ via its mean and covariance operator. In particular, $X_H$ has properties that make it "nice" to deal with, e.g. it leads to nice calculations with means and covariances, and connects abstract covariance operators to the covariance of $X$. For example, it is not hard to show that if $C_X$ is the covariance operator of $X_H$, then $$ \text{cov}(f(X)) = \langle f, C_Xf\rangle. $$
My question: Is there a similar construction that holds for arbitrary Hilbert spaces, or say even Banach spaces? Another way to think of this: Is there a way to embed $X$ into an RKHS $H$ that doesn't explicitly involve the kernel $k$ (but still leads to "nice" properties)?
(I am aware that one could define many different types of embeddings ad hoc, but my question is geared more towards whether or not there is a canonical way to do this that leads to something similar to $k(X,\cdot)$ and if so, what type of results are out there.)
