You can think about attaching a solid torus as a two-step process:
First, attach $D^2 \times I$ where $D^2$ is the meridional disc of the attaching torus.
Secondly, attach the remaining of the solid torus, which is homeomorphic to a ball $B^3$.
Now if we know where the meridian of the solid torus goes, then we know the result of the first step, $M_1$, up to isotopy. On the other hand, the result of the second step is determined by the map $\phi \colon \partial B^3 \rightarrow \partial M_1$, up to isotopy. By Smale's theorem, any orientation-preserving homeomorphismdiffeomorphism of the 2-sphere is smoothly isotopic to the identity map. Hence the second step is essentially done uniquely.