The claim you wrote down is incorrect. The stabilizer of $p$ is simply $ \mathrm{SO} \left( n-2 \right) $. The intuition is that in order to stabilize $p$, your matrix $f$ must be a block matrix of the form: $$ f = \begin{bmatrix} I_2 & 0 \\ 0 & U \end{bmatrix} $$ where $U \in \mathrm{SO} \left( n-2 \right)$. This is because we must map the first two coordinates to themselves, but we can do any linear map on the $n-2$ remaining ones and they would still map to zero (this is not a complete proof but I hope you get the idea).

The Lorentz group does not include translations, so you have no where to get the $\mathbb{R}^{n-2}$ factor from. Perhaps you meant to start with the Poincaré group rather than the Lorentz group. The Poincaré group is: $$ \mathrm{SO} \left( n-1, 1 \right) \ltimes \mathbb{R}^n $$ so essentially it includes Lorentz transformations as well as translations. However, the stabilizer of $p$ still does not include translations, as those would shift the last $n-2$ coordinates away from zero.

One may prove by direct computation that in order to stabilize $p$, your matrix $f$ must be a block matrix of the form: $$ f = \begin{bmatrix} 1 & 0 & \vec{v}^T \\ 0 & 1 & -\vec{v}^T \\ && U \end{bmatrix} $$ where $\vec{v} \in \mathbb{R}^{n-2}$ and $U \in \mathrm{SO} \left( n-2 \right)$. This is because we must map the first two coordinates to themselves, but we can do any linear map on the $n-2$ remaining ones and they would still map to zero; and multiplying those coordinates by any vector $ \vec{v}^T $ still won't do anything, since they are zero. The group of all matrices of the above form is identified with the Euclidean group that you wrote down.