In addition to all the completely correct answers to this question, it's probably worth spelling out some of the misconceptions that are common to physicists who work in this area and giving the physics answer.

The main thing that arises all the time is that physicists often conflate representations of a group and of its Lie algebra. The reason one can get away with this is that really in physics one deals with projective representations which are the same (in most common cases) as representations of the universal cover of the group. Thus, distinctions like the difference between $Spin(4) \simeq SU(2) \times SU(2)$ and $SO(4) \simeq \left(SU(2) \times SU(2)\right) / \mathbb{Z}_2$ are lost.

The other slightly odd (and not so common) misconception that seems to be here is that you need a group to be a subgroup of $U(n)$ to have a complex $n$ dimensional representation. A map into $U(n)$ is more than enough for obvious reasons.

Taking these into account, the original statement in the question becomes that you have a map from $Spin(5) \simeq Sp(4) \to SU(4)$ giving a four dimensional representation. And the answer to the physics version of the question is "yes-ish", because there is a map from $Spin(4) \to SU(2)$ given by projection onto one of the two $SU(2)$ components (say) and then you can take the $\mathbb{Z}_2$ quotient to get a map from $SO(4) \to SO(3)$.

Or, to put it another way, there is a perfectly cromulent, but not faithful, three dimensional representation of $SO(4)$.