Try "Measure, integral and probability", too. It doesn't cover all the topics you mentioned, but you should be able to work through it quite quickly, and it will establish a good base to build on.

Hitchhiker's guide is considerably more technical than Williams. It's very complete, but there's a danger of information overload there.

Jacod and Protter's book is a reasonably friendly introduction, and doesn't require any background beyond multivariable calculus. As camomille said, you're looking at a couple of months of work to take all this stuff on board.

None of the references were as systematic as I would have liked, but I largely stopped pursuing this line of inquiry when you pointed out that the laws of the integrated processes are still singular.

Thanks for your answer Tom. I don't have much intuition about Banach spaces and their duals. Is that a restrictive assumption? Can one always find such a measure $\mathbb{P}$?

Perhaps "in most textbooks" was a little strong. Some books do start with a non-negative definite matrix $\Sigma$ and then specify the density function. I expect this construction is much more common in applied maths. I've checked two textbooks I had to hand - one engineering text and one on machine learning. They both construct the mutlivariate normal in this way.

Yes, that's more or less equivalent to constructing a Brownian motion on the space. You need some generalisation of the Laplace operator.

Thanks for your comment. The construction in Nualart that you mentioned is for a set of one-dimensional Gaussians indexed by a Hilbert space. I'm really looking for analogues of the normal distribution that take values in a space $S$.

I guess you need $T$ to be seperable, right? Otherwise you can't define a process by specifying its finite dimenstional distributions.

Maud: "Fearless Symmetry" by Ash and Gross is a non-technical introduction to the subject.

Persi Diaconis mentioned a similar problem viewed from a slightly different perspective in his paper "The Markov chain Monte-Carlo revolution"