Let $X$ be a smooth projective connected complex algebraic variety with ample canonical bundle. Let $h$ be the hilbert polynomial of the canonical bundle.

Why is the moduli stack of canonically polarized varieties with hilbert polynomial $h$ of dimension $>0$? (I know that it could have a zero-dimensional connected component, think of a fake projective plane.)

In other words, I'd like to know why the existence of $X$ forces the existence of many varieties that are "like" X.