Yes, this is true. This answer is essentially an expanded version of Jason Starr's comment, also including a reference.
The result that we need is the following, that can be easily deduced from [G. M. Greuel, C. Lossen, E. Schustin, Introduction to singularities and deformations, Theorem 2.6 Chapter I].
Proposition. Let $f \colon \mathcal{X} \to \Delta$ be a deformation of the affine hypersurface $X=f^{-1}(0)$ over a disk. Assume that $0$ is the only singular point for $X$ and that $X_t$ has only isolated singularities. Then the total Milnor number of the fibres is upper semicontinuous, i.e. for all $t \in \Delta$ we have $$\mu(X, \, 0) \geq \sum_{x \in \textrm{Sing}(X_t)} \mu(X_t, \, x).$$ In other words, the total Milnor number cannot decrease under specialization.
In your case, the total Milnor number of every hypersurface in $\mathscr A$ is at least $k$, whereas the total Milnor number of the hypersurface $\{p=0\}$ is the number of its nodes, hence strictly less than $k$.
Therefore, $\{p=0\}$ is not a specialization of hypersurfaces belonging to $\mathscr A$.