I'm looking for a precise reference for the following theorem.
Let $C$ be a smooth curve over $\mathbb{C}$ and let $S$ be a finite set of closed points of $C$. Let $\ V$ be a polarized variation of $\mathbb{Z}$-Hodge structures over $C\setminus S$. Fix $b$ in $C\setminus S$.
Theorem. Assume that the local system $\pi_1(C\setminus S)\to \mathrm{Aut}( V_b)$ factors via $\pi_1(C)$. (In other words, the local system extends.) Then $ V$ extends to a polarized variation of $\mathbb Z$-Hodge structures on $C$.
Equivalently (but less precise):
Theorem. Assume a local system on $C$ comes from a variation of Hodge structures on $C\setminus S$. Then the local system on $C$ comes from a variation of Hodge structures on $C$ (which has to be compatible with the one on $C\setminus S$).