This follows from the case of graphs (i.e. hypergraphs where all sets have size $\leq 2$). As I explained in the comments above, Hadwiger's conjecture says that a graph with no $K_k$ minor is $(k-1)$-colorable, so the current question for graphs is the $k=5$ case of Hadwiger's conjecture. In 1937, Wagner showed that the $k=5$ case of Hadwiger for graphs is equivalent to the $4$-color theorem; in 1976, Appel and Haken proved the $4$-color theorem. So the OP's conjecture is true for graphs, and now we need to prove the hypergraph case.
Let $H$ be a hypergraph with ground set $V$. Define $\Gamma(H)$ to be the graph with vertex set $V$ where $(i,j)$ is an edge of $\Gamma(H)$ if and only if there is some set $E$ in $H$ with $\{i,j \} \subseteq E$. The following lemmas are immediate from the definitions:
Lemma For any subset $W$ of $V$, $\Gamma(H|_W) = \Gamma(H)|_W$.
Lemma The multigraphhypergraph $H$ is connected if and only if the graph $\Gamma(H)$ is connected.
Lemma $S$ and $T$ are connected to each other in $H$ if and only if there is an edge from $S$ to $T$ in $\Gamma(H)$.
Combining these lemmas, $H$ has a $K_5$ minor if and only if $\Gamma(H)$ has a $K_5$ minor. Now, suppose that $H$ has no $K_5$ minor, so that $\Gamma(H)$ has no $K_5$ minor. By this case of Hadwiger's conjecture, the graph $\Gamma(H)$ is $4$-colorable; let $\chi : V \to \{ 0,1,2,3 \}$ be a $4$-coloring. Then I claim that $\chi$ is also a $4$-coloring of $H$.
Indeed, let $E$ be any edge of $H$ with $|E| \geq 2$, and let $i$, $j$ be two distinct elements of $E$. Then $(i,j)$ is an edge of $\Gamma(H$), so $\chi(i) \neq \chi(j)$, so $\chi : E \to \{0,1,2,3 \}$ is not constant, as desired. QED
The other direction of the last step doesn't work -- a coloring of $H$ doesn't always induce a coloring of $\Gamma(H)$. But, fortunately, that's not the direction we need.
