Fortunately, these questions are easy to answer. First of all, it helps to distinguish between polarizable Hodge structures and polarizaedpolarized structures. For polarizable, we merely require that a polarization exists, but it is not fixed. ThenLet Hodge structure mean pure rational Hodge structure below. Then
The category of polarizable pure Hodge structures is abelian and semisimple (morphisms are not required to respect polarizations). This is essentially proved in Theorie de Hodge II.
The category of arbitrary Hodge structures is abelian but not semisimple. To see the nonsemisimplicity, we can use a theorem of Oort-Zarhin [Endmorphism algebras of complex tori, Math Ann 1995], that any finite dimensional $\mathbb{Q}$-algebra is the endomorphism algebra of some complex torus, and therefore of some Hodge structure.
All pure Hodge structures of geometric origin are polarizable. So this is a very reasonable condition to impose. For $Gr_WH^*(X)$, this is explained for example in Beilinson's Notes on absolute Hodge cohomology (although this was already implicit in Deligne's construction).