As long as connected groups of isometries are concerned, Grassmann manifolds are symmetric spaces, so the identity component of its isometry group is $G$ in its symmetric presentation $G/H$ ($G$ connected) as a homogeneous space, namely, $SO(n)$ for $n$ odd and $SO(n)/\mathbf Z_2$ for $n$ even in the real case, and $PU(n)=SU(n)/\mathbf Z_n$ in the complex case. (Note that $U(n)$ acts on the left on the Grassmannian with a $U(1)$-kernel (its center), so the effectivized group is the projectivization $PU(n)$. Moreover the center of $U(n)$ meets $SU(n)\subset U(n)$ along its center, which consists of $\omega I$ where $\omega$ is an $n$-th root of unit.)
Further, Cartan described the full isometry groups of symmetric spaces, and an explicit result is easy to figure out in the case of Grassmann manifolds. I do not remember now, but you can find Cartan's description in the book of O. Loos on symmetric spaces, the second volume. I tend to agree with Ryan when he writes that in the case of Grassmann manifolds, the full isometry group should be $G\times N_G(H)$.
About Stiefel manifolds: with the metric you describe, they are normal homogeneous spaces $G/H$, i. e. have the metric induced from a bi-invariant Riemannian metric on $G$. There is a recent paper by S. Reggiani with a very effective way of computing the identity component of isometry groups of normal homogeneous spaces in here.
Since Stiefel manifolds fiber over Grassmann manifolds,
Added: I think it shouldn't be very hard to use this fiber bundle to figure out their looked up Loos, "Symmetric spaces, II", Theorem 4.4 and the ensuing Table 10 on page 156 for the full isometry group of the real and complex Grassmannians. If I understand correctly, indeed in the case of complex Grassmannians
$SU(n)/S(U(p)\times U(n-p))$, every isometry comes from left multiplication by elements from $SU(n)$ except for two cases: an isometry induced by complex conjugation; and mapping a $p$-plane to its orthogonal complement in case $n=2p\geq4$. In the case of
real unoriented Grassmannians $SO(n)/S(O(p)\times O(n-p))$, every isometry comes
from left multiplication by an element of $O(n)$ except for: mapping a $p$-plane to its orthogonal complement in case $n=2p\geq4$; the symmetric group $S_3$ in case $n=2p=8$,
coming from outer automorphisms of $\mathfrak{so}(8)$.
