I am interpreting the question as also implying that $X$ itself is not a product. I do not have an answer in general, but I think that I have an example. It is a projective surface with a singular point: since you did not ask for the variety $X$ to be non-singular, this should qualify! Probably if $X$ is a surface with your property, then it must be singular, but I do not know how to show this.

Let $C$ be a non-hyperelliptic curve of genus at least three and let $Y := C \times C$. The diagonal $D$ in $Y$ is a curve that can be contracted: there is a morphism $\varphi \colon Y \to {\rm Jac}(C)$ given by $(x,y) \mapsto ([x]-[y])$. The morphism $\varphi$ contracts only the diagonal $D$, since the curve has genus at least three and it is non-hyperelliptic. The image of $\varphi$ is the surface $X$ we are looking for. It is a projective surface (with a unique singular point); its blow up is $Y$ and is therefore a product.

Presumably, the assumption that the curve is non-hyperlliptic is irrelevant: instead of letting $X$ be the image of $\varphi$, you can probably define $X$ to be the "intermediate" surface in the Stein factorization of $\varphi$.