Finally, I was able to cook up a counterexample. We choose $X = c_0$ (zero sequences equipped with supremum norm). Thus, the dual spaces are (isometric to) $X^* = \ell^1$ and $X^{**} = \ell^\infty$.
We define
$$
C := \{ x \in \ell^1 \mid \forall n \in \mathbb N : |x_n| \le 1/n^2 \}$$
and $f \colon \ell^1 \to \mathbb R \cup \{\infty\}$ via
$$
f(x) = \sum_{n=1}^\infty x_n \in \mathbb R
$$
for all $x \in C$ and $f(x) = \infty$ for all $x \in \ell^1 \setminus C$.
Let us check, that the assumptions are satisfied. The set $C$ is bounded due to $\sum_{n = 1}^\infty 1/n^2 < \infty$ and weak-$\star$ closed since it is the intersection of the weak-$\star$ closed "stripes"
$$ \{x \in \ell^1 \mid |x_n| \le 1/n^2\} \qquad\forall n \in \mathbb N.$$
Thus, it is weak-$\star$ compact.
The function $f$ is convex and it remains to check weak-$\star$ continuity on $C$. Let $x_0 \in C$ be given and consider a net $(x_i)_{i\in I} \subset C$ with $x_i \to x_0$.
For an arbitrary $\varepsilon > 0$, there is $N \in \mathbb N$ with
$\sum_{n = N+1}^\infty 1/n^2 < \varepsilon$.
Next, there is $i \in I$ with
$$
\left| \sum_{n = 1}^N (x_{j,n} - x_{0,n}) \right| < \varepsilon
\qquad\forall j \ge i$$
since $y \mapsto \sum_{n = 1}^N y_n$ is weak-$\star$ continuous.
Thus,
$$
|f(x_j) - f(x_0)|
\le
\left| \sum_{n = 1}^N (x_{j,n} - x_{0,n}) \right|
+
\sum_{n = N+1}^\infty |x_{j,n}|
+
\sum_{n = N+1}^\infty |x_{0,n}|
<
3 \varepsilon
\qquad\forall j \ge i.$$
Since $\varepsilon > 0$ was arbitrary, this shows weak-$\star$ continuity on $C$.
Finally, it is easy to check that $\partial f(0) = \{1\}$, but $1 \in \ell^\infty \setminus c_0$.
