Let me summarize comments.

 1. abx points out that $Q$ should be integer valued, otherwise it's not an interesting notion.
 2. (Assuming integrality) the answer is no because the categories of Hodge structures of type $\{(1,0), (0,1)\}$ and complex tori are equivalent. The polarizable ones correspond to abelian varieties, which is a proper subset.