I'm not familiar with equating measures to functions. Do you mean that $\hat p$ is half a Lebesgue measure on $[0,1]$ with another half-mass at $0$?

Thanks, not sure I understand the notation here. What is a $1_{[0,1]}$ in definition of $\hat p$, a Lebesgue measure? You also a similar notation below $1_{\{0\}}$, where I guess it means the indicator function instead.

I mean something of the kind "for $\lambda$ small enough, the optimal solution is $F = \Bbb E[Z|Y] + ...$"

I looked into similar (more general) problem in this paper some time ago. For general Markovian dynamics I applied Lyapunov-like functions, there are a couple of references in the text on how to find them. In your case it may be harder unless you have a positive drift, however I think you can use the fact that $\gamma > 1$ to compensate for it for $X$ large enough (where you need the convergence to infinity).

+1 for mentioning the Netherlands

I would not expect as elegant results in the penalized case: the original minimization problem (with $\lambda = 0$) is quadratic, optimizer is linear and can be nicely characterized as a projection of $Z$ onto $\sigma(Y)$. I'd expect you can get some local deviation results for $\lambda \ll 1$, however not sure whether you'd find them useful.

@UriBader: it's tricky to speak about convexity here - see related MSE question - there it is quite crucial which convexity is used.

@UriBader: thanks, have a nice flight. Having $P$ being all non-delta measures does not provide a counterexample to the OP though (if that's what you've meant). Let's say $\hat p = \delta(0)$, then taking $f = 1_{\{0\}}$ means that $\hat p f = 1$ but $p f = 0$ for all $p\in P$.

@UriBader: can you be more specific, please? It is indeed a standard Borel space

You are right, I'm used to think in terms of integrals.

Thanks, in your example - what if $M$ can be represented as a combination of some other elements w.r.t. general measure, not just finite linear combination? I highly doubt that, but unfortunately in that case we can't really easily use dominance, hence I am not sure how would you prove that.

What's a problem of finding derivative for intervals where $M$ is constant, and adding jumps as a separate sum?

So what's the problem with existence of family, if its elements do exist?