Here's a counter-example.

Let $\mu$ be counting measure supported on $\mathbb Z$; so $\int f(x) \ d\mu(x) = \sum_{m\in\mathbb Z} f(m)$ for $f$ continuous with compact support.

Choose a very rapidly decreasing sequence of positive reals $\delta_n$.
Let $\psi$ be the piecewise linear function which is $1$ at $n+1/n$ for $n\geq 10$ (say), and is $0$ at $n+1/n \pm \delta_n$ (and is zero at all $x<10+1/10-\delta_{10}$). By definition, $\psi$ is continuous (and could be made smooth by the use of a bump function) and $\psi$ is Lebesgue integrable (if $\delta_n$ decreases fast enough).

Then set
`\[ \alpha(x) = (\psi*\mu)(x) = \int \psi(x-y) \ d\mu(y) = \sum_{m\in\mathbb Z} \psi(x+m). \]`

A priori, this sum might diverge, but only to $+\infty$. Clearly $\alpha$ is periodic in that $\alpha(x+k)=\alpha(x)$ for any $k\in\mathbb Z$.

Fix $x\in[-1/2,1/2)$, and consider which $m\in\mathbb Z$ are such that $\psi(x+m)>0$. This occurs iff there is $n\geq 10$ with $1/n-\delta_n < x+m-n < 1/n+\delta_n$. As $m-n\in\mathbb Z$ and $n\geq 10$ and $|x|\leq 1/2$, this can only occur if $m=n$. So $1/n-\delta_n < x < 1/n+\delta_n$. Choosing $(\delta_n)$ suitably, we can arrange that $1/(k+1)+\delta_{k+1} < 1/k-\delta_k$ for all $k$, and then $n$ is unique for any given $x$.

We conclude that for every $x$, there is at most one $m\in\mathbb Z$ with $\psi(x+m)>0$. In particular, $\alpha(x)\in[0,1]$ for all $x$.

However, clearly $\alpha(1/n)\geq 1$ for all $n\geq 10$, but $\alpha(0) = 0$ as if $\psi(m)>0$ for some $m\in\mathbb Z$, then there is $n\geq 10$ with $-\delta_n < m-n-1/n < \delta_n$ which forces $m=n$, but then $-\delta_n <-1/n<\delta_n$, which we can avoid by a suitable choice of $(\delta_n)$. So $\alpha = \psi*\mu$ is not continuous at $0$.

define$\psi*\mu$?? Because this seems to be, formally, $x\mapsto \int \psi(x-y) \ d\mu(y)$. But to make sense of this integral, don't we need that $y\mapsto \psi(x-y)$ is in $L^1(\mu)$? Or is this sort of implicit in (3)? – Matthew Daws Sep 22 '11 at 10:52