Is $g(v)=\mathbb{E}[f(v+W)]$ a differentiable function of $v$ when $f$ is continuous and $W$ is multivariate normal? Suppose $f$ is a continuous function on $\mathbb{R}^n$, and $W$ has a multivariate normal distribution on $\mathbb{R}^n$. If the expectation
$$g(v)=\mathbb{E}[f(v+W)]$$
is defined for all $v \in \mathbb{R}^n$, must it also be differentiable for all $v$? Or what other sufficient condition on $f$ would make this expectation differentiable?
I suspect that continuity (or maybe measurability) is sufficient, so that the answer to the first question is yes. This may follow from the smoothness of the Weierstrass transform, but I haven't found a statement of that smoothness with a clear condition on $f$, or with a proof explicit enough for me to see what to do in $n$ dimensions. Perhaps there's a good reference on this that I haven't found.
A non-trivial example with $n=1$ is $f(x)=\sqrt{|x|}$, where I can use Mathematica to write $g$ as a messy combination of Bessel functions, and to verify that $g$ is differentiable even at $v=0$. So I'd want some sufficient condition which allows $f$ and its derivatives to be unbounded.
 A: I assume that $W$ has a non-degenerate normal distribution. In this case, by an appropriate change of variables, we may assume that $W$ has the standard normal distribution, so that $g$ is the (absolutely convergent) convolution of $f$ and $h(x) = (2\pi)^{-d/2} \exp(-|x|^2/2)$. Our goal is to show that $g = f * h$ is differentiable. Clearly, $h$ is differentiable, and we will apply dominated convergence theorem in order to show that $\nabla g = f * \nabla h$. More precisely, we will pass to the limit as $y \to 0$ under the integral sign in
$$ \begin{aligned} & \frac{f * h(x + y) - f * h(x) - y \cdot (f * \nabla h)(x)}{|y|} \\ & \qquad = \int_{\mathbb R^d} f(x - z) \, \frac{h(z + y) - h(z) - y \cdot \nabla h(z)}{|y|} \, dz \\ & \qquad = \int_{\mathbb R^d} f(x - z) \, \frac{y \cdot \nabla h(\zeta_{z,y}) - y \cdot \nabla h(z)}{|y|} \, dz ; \end{aligned} \tag{$\spadesuit$} $$
in the last step, we used the mean value theorem, and of course $\zeta_{z,y}$ lies on the interval $[z,z+y]$. The integrand in the right-hand side converges pointwise to zero, and we only need to show that dominated convergence theorem applies.

Let $\phi(x)= (2\pi)^{-d/2} \exp(-|x|^2/2)$ (which is the same as the formula for $h$, but with $\phi:\mathbb{R}\to\mathbb{R}$ and $h:\mathbb{R}^n\to\mathbb{R})$. Observe that $\nabla h(x) = -\phi(x) x$. Then is easy to see that
$$ |t|\, \phi(t) \leqslant \phi(t+1) + \phi(t-1),$$
and hence whenever $|s| \leqslant 1$ we have
$$ \begin{aligned} |t + s|\, \phi(t+s) & \leqslant \phi(t+s+1) + \phi(t+s-1) \\ & \leqslant \phi(t+2) + \phi(t) + \phi(t-2). \end{aligned} $$
It follows that
$$ \sup_{y : |y| \leqslant 1} |\nabla h(x + y)| \leqslant \sum_e h(x + e) , $$
where the summation spans over all vectors $e$ with entries $-2, 0, 2$. In particular, if $|y| \leqslant 1$, then
$$ \biggl| \frac{y \cdot \nabla h(\zeta_{z,y}) - y \cdot \nabla h(z)}{|y|} \biggr| \leqslant |\nabla h(\zeta_{z,y})| + |\nabla h(z)| \le 2 \sum_e h(z + e) $$
Since the convolution of $f$ and $h(\cdot + e)$ is absolutely convergent for every $e$, we may indeed apply dominated convergence theorem to the right-hand side of ($\spadesuit$), as desired.
