If the variety X is defined over C, is compact and smooth, then it should be possible to make it rigid. Indeed, the variety Aut(X) has at most countable number of irreducible components and all these components should be of bounded dimension N.
Let us prove it will be sufficient to fix N points. For every component Comp of Aut(X) let us consider the subvariety of X that is mooved by at least one element of this component Comp. It is clear that mooved point will be an open subset of X, so all of them will have an intersection (a countable intersection of open subsets is non-empty). So there will be a point on X that is mooved by at least on element from every component of Aut(X). Let us mark this point, call it P. Note now that the components of Aut(x,P) have dimenions at most N-1. So we procede by induction.

In fact we did not really used smoothness of X but we suerlly used that X is defined over C.

If the variety $X$ is defined over $\mathbb C$, is compact and smooth, then it should be possible to make it rigid. Indeed, the variety $Aut(X)$ has at most countable number of irreducible components and all these components should be of bounded dimension $N$.

Let us prove it will be sufficient to fix $N+1$ points (where was $N$ here before the comment of Brian). For every component Comp of $Aut(X)$ let us consider the subvariety of $X$ that is moved by at least one element of this component Comp. It is clear that moved point will be an open subset of $X$, so all of them will have an intersection (a countable intersection of open subsets is non-empty). So there will be a point on $X$ that is moved by at least on element from every component of $Aut(X)$. Let us mark this point, call it $P$. Note now that the components of $Aut(X,P)$ have dimensions at most $N-1$. So we proceed by induction.

In fact we did not really used smoothness of $X$ but we surely used that $X$ is defined over $\mathbb C$.