A topological remark:
If $E \subset Y$ is a closed analytic subspace of a smooth space $X$, then the boundary of a tubular neighbourhood is an (odd dimensional, real) sphere bundle over $E$.
Thus if the blowup of $Z \subset X$ is smooth then the boundary of a tubular neighbourhood of $Z \subset X$ can be written as a sphere bundle over the exceptional divisor $E = P(N_Z X)$.
One might use this to arrive at a contradiction e.g. by calculating cohomology.
Similar ideas were used by Mumford in his study of normal surface singularities. Mumford: The topology of normal singularities of an algebraic surface and a criterion for simplicity.