If you are considering compact 3-manifolds, then it is conjectured that the fundamental groups are always linear, so there should be no restriction on the topology. One may as well consider 3-manifolds with indecomposable fundamental group. Then the only remaining case to consider is graph manifolds with a non-trivial JSJ decomposition and which do not admit a non-positively curved Riemannian metric. See the papers of Yi Liu and Przytycki-Wise.
For noncompact 3-manifolds, the issue of linearity of the fundamental group is a wide-open problem. I don't know of any evidence against it though.
