Consider the action of $\operatorname{GL}(\mathbb{R}^4)$ on the Grassmannian of 2-dimensional subspaces of $\mathbb{R}^4$. In experiments, I observe that four randomly drawn points in this space are simultaneously stabilized by nontrivial members of $\operatorname{GL}(\mathbb{R}^4)$. This violates my naive attempts at counting dimensions:

The stabilizer subgroup of any given plane is a generic subset of a 12-dimensional subspace of $\mathbb{R}^{4\times 4}$. If I draw three planes at random, the intersection of the corresponding subspaces is 4-dimensional, as expected. If I draw a fourth plane at random, I expect the resulting intersection to be the span of the identity matrix, but alas, I consistently obtain a 2-dimensional intersection. Only after I draw a fifth plane does the resulting intersection equal the span of the identity matrix.

What's going on here? More generally, when should I expect subsets of Grassmannian spaces to have trivial stabilizer?