Let us assume, as Beauville does in his book, that the surface $S$ is smooth. The exceptional divisor $E$ is thus isomorphic to $\mathbb P^1$, a smooth divisor on the smooth surface $\hat{S}$. The inequality actually holds point by point, so fix $x\in C'\cap E$. Since multiplicities are a local invariant, we work in the completed local ring of $\hat{S}$ at $x$, which is isomorphic to $\mathbb C[[s,y]]$. We may even choose coordinates so that $E$ is locally cut out by $(y)$, and $C'$ by $f\in (s,y)\subset \mathbb C[s,y]$. Since $C'$ is the strict transform, it shares no component with $E$, so $f\notin (y)$.

By $m_x(C'\cap E)$, he means the length of the scheme-theoretic intersection, which is a 0-dimensional scheme. If $C'$ is locally cut out by $f\in \mathbb C[[s,y]]$, then
$$m_x(C'\cap E) = \dim \mathbb C[[s,y]]/(y,f)$$
which is the minimal degree of a monomial in $f$ not divisible by $y$. By $m_x(C')$, he means the degree of $f\in \mathbb C[s,y]$, that is the minimal degree of a monomial in $f$. Thus,
$$m_x(C')\leq m_x(C'\cap E).$$
For the example, you are correct in taking $C$ to be a cuspidal curve, say $V(y^2-t^3)\subset \mathbb A^2_{(t,y)}$. In the blow up, $C'$ is cut out by $s^2-y$ on the appropriate affine patch of $\hat{S}$. Here, $m_x(C'\cap E)=2$ whereas $m_x(C')=1$.