Edit. There was a mistake in the original answer, arising from multiple fibers. I added a hypothesis that eliminates multiple fibers. I also added some examples at the end explaining what goes wrong if the hypothesis fails.
Fix an ample divisor class $H$ on $X$. Each curve $Z_y$ has finite $H$-degree.
Notation 1. Denote by $e$ be the minimal integer such that for infinitely many values $y\in Y$, the curve $Z_y$ has $H$-degree $e$, and denote by $Y_e\subset Y$ the set of such $y$. For each pair $(y,y')\in Y \times Y \setminus \Delta(Y)$, denote by $m_{y,y'}$ the minimum integer $m\geq 0$ such that the projections of $Z_y\times_X Z_{y'}$ to $Z_y$ and to $Z_{y'}$ factor through the Cartier divisor $m\cdot\underline{p_0}\subset Z_y$, resp. $m\cdot \underline{p_0}\subset Z_y$. Since intersection multiplicity is an upper semicontinuous function, and since the parameter space of curves of $H$-degree $e$ is of finite type, the set of integers $m_{y,y'}$ is bounded above. Denote by $m_e$ the maximum integer such that $m_{y,y'}$ equals $m_e$ for infinitely many pairs $(y,y')$ in $Y_e\times Y_e\setminus \Delta(Y_e)$.
Additional Hypothesis. For every $y\in Y$, the $H$-degree of $Z_y$ is not a proper divisor of $e$.
Main Result. Under the additional hypothesis, there exists an integer $d\geq 1$ and an isomorphism of $X$ with the contraction of the unique contractible cross-section of the projection $\mathbb{P}(\mathcal{O}_{\mathbb{P}^1}\oplus \mathcal{O}_{\mathbb{P}^1}(-d))\to \mathbb{P}^1,$ i.e., $X$ is isomorphic to a projective cone over a rational normal curve.
Definition 2. A normal modification over $p_0$ is a projective, birational morphism, $\nu:\widetilde{X}\to X$, such that $\widetilde{X}$ is normal and whose exceptional locus $E$ maps to $\{p_0\}$. For every $y\in Y$, the strict transform of $Z_y$ with respect to $\nu$, denoted $\widetilde{Z}_y$, is the closure in $\widetilde{X}$ of $\nu^{-1}(Z_y\setminus\{p_0\})$. For every $(y,y')\in Y\times Y\setminus \Delta(Y)$, denote by $\widetilde{m}_{y,y'}$ the minimum integer $m\geq 0$ such that the projections of $\widetilde{Z}_y\times_{\widetilde{X}}\widetilde{Z}_{y'}$ to $\widetilde{Z}_y$ and to $\widetilde{Z}_{y'}$ factor through a Cartier divisor of length $m$ in $\widetilde{Z}_y$, resp. of length $m$ in $\widetilde{Z}_{y'}$.
Lemma 3. For every $y\in Y$, the curve $\widetilde{Z}_y$ is isomorphic to $\mathbb{P}^1$, and the intersection of $\widetilde{Z}_y$ and $E$ is a singleton set $\widetilde{p}_y$. Moreover, for every pair $(y,y')\in Y\times Y\setminus \Delta(Y),$ the integer $\widetilde{m}_{y,y'}$ is bounded above by $m_{y,y'}$.
Proof. Since $Z_y\setminus\{p_0\}$ is isomorphic to $\mathbb{A}^1$, also $\nu^{-1}(\widetilde{Z}_y\setminus\{p_0\})$ is isomorphic to $\mathbb{A}^1$. Thus the normalization of $\widetilde{Z}_y$ is isomorphic to $\mathbb{P}^1$. Thus, $\widetilde{Z}_y\cap E$ is set-theoretically isomorphic to $\mathbb{P}^1\setminus \mathbb{A}^1$, i.e., a singleton set. If the morphism from $\mathbb{P}^1$ to $\widetilde{Z}_y$ is ramified at this singleton set, then the same holds for the composition of this morphism with the birational homeomorphism $\nu_{\widetilde{Z}_y}:\widetilde{Z}_y\to Z_y$. By hypothesis, $Z_y$ is isomorphic to $\mathbb{P}^1$. Thus, also $\widetilde{Z}_y$ is isomorphic to $\mathbb{P}^1$.
By the previous paragraph, the projections $\widetilde{Z}_y\to Z_y$ and $\widetilde{Z}_{y'}\to Z_{y'}$ are isomorphisms. Also the projection of $\widetilde{Z}_y\times_{\widetilde{X}} \widetilde{Z}_{y'}$ to $Z_y\times Z_{y'}$ factors through $Z_y\times_X Z_{y'}$. Thus, the image of $\widetilde{Z}_y\times_{\widetilde{X}} \widetilde{Z}_{y'}$ in $Z_y$, resp. in $Z_{y'}$, is contained in the Cartier divisor $m_{y,y'}\cdot \underline{p_0}$, resp. $m_{y,y'}\cdot \underline{p_0}$. Thus, $\widetilde{m}_{y,y'}$ is bounded above by $m_{y,y'}$. QED
Proposition 4. There exists a (minimal) normal modification $\nu:\widetilde{X}\to X$ such that for an infinite subset $Y'_e\subset Y_e$, the strict transforms $\widetilde{Z}_y$ are pairwise disjoint for all $y\in Y'_e$.
Proof. This is constructed by induction on $m_e$. Let $\nu_1:\widetilde{X}_1\to X$ be the normalization of the blowing up of $X$ at the reduced scheme $\{p_0\}$. Every (non-isomorphic) modification of $X$ over $p_0$ factors unique through this blowing up. If the modification is normal, then it factors uniquely through the normalization of this blowing up. Thus, every (non-isomorphic) normal modification of $X$ over $p_0$ factors uniquely through $\nu_1$.
By a standard blowing up computation (locally embed $X$ in affine space, and use the standard coordinates on the blowing up of $\{p_0\}$ in affine space), the intersection $\widetilde{Z}_y\times_{\widetilde{X}} \widetilde{Z}_{y'}$ has strictly smaller intersection multiplicity $\widetilde{m}_{y,y'}$ than $m_{y,y'}$. In the base case that $m_e$ equals $1$, for those $(y,y')$ with $m_{y,y'}$ equal to $1$ (all but finitely many such $(y,y')$, the strict transforms $\widetilde{Z}_y$ and $\widetilde{Z}_{y'}$ are disjoint.
In case $\widetilde{Z}_y$ and $\widetilde{Z}_{y'}$ are not already disjoint for infinitely many $(y,y')\in Y_e\times Y_e\setminus \Delta(Y_e)$, define $\widetilde{m}_e$ to be the minimum integer such that $\widetilde{m}_{y,y'}\leq \widetilde{m}_e$ for all but finitely many pairs $(y,y')\in Y_e\times Y_e\setminus \Delta(Y_e)$. The set of singleton sets $\widetilde{Z}_y\cap E$ takes only finitely many values among singletons in $E$, or else already $\widetilde{Z}_y\cap \widetilde{Z}_{y'}$ is empty for infinitely many $(y,y')\in Y_e\times Y_e\setminus \Delta(Y_e)$. Thus, there exists $p_1\in E$ such that for infinitely many pairs $(y,y')\in Y_e\times Y_e\setminus \Delta(Y_e)$, the intersection $\widetilde{Z}_y\cap \widetilde{Z}_{y'}$ equals $\{p_1\}$. Denote by $Y_{e,p_1}$ the infinite set of $y\in Y_e$ with $\widetilde{Z}_y\cap E$ equal to $\{p_1\}$.
Since $\widetilde{m}_e$ is strictly less than $m_e$, by the induction hypothesis, there exists a minimal normal modification over $p_1$, $\nu':\widetilde{X}\to \widetilde{X}_1$, such that for infinitely many pairs $(y,y')\in Y_{e,p_1}\times Y_{e,p_1}\setminus \Delta(Y_{e,p_1})$, the strict transforms of $Z_y$ and $Z_{y'}$ in $\widetilde{X}$ are disjoint. Thus the composition $\nu=\nu_1\circ \nu'$ is the unique minimal, normal modification such that the strict transforms of $Z_y$ and $Z_{y'}$ are disjoint for infinitely many pairs $(y,y')$ in $Y_e\times Y_e\setminus \Delta(Y_e)$. QED
Proposition 5. For $\nu$ as above, there is a Cartier divisor class on $\widetilde{X}$ of self-intersection number $0$ whose linear system is a pencil $\Pi$, and such that for infinitely many $y\in Y'_e$, the strict transform $\widetilde{Z}_e$ is a smooth Cartier divisor in this pencil.
Proof. Since $\widetilde{X}$ is normal, there are only finitely many singular points. Thus, among the infinitely many, pairwise-disjoint curves $\widetilde{Z}_y$ with $y\in Y'_e$, at most finitely many of these contain singular points of $\widetilde{X}$. After removing these from $Y'_e$, the curve $\widetilde{Z}_y$ is a smooth curve, isomorphic to $\mathbb{P}^1$, contained in the smooth locus of $\widetilde{X}$ for all $y\in Y'_e$. In particular, it is a Cartier divisor.
There are only finitely many Cartier divisor classes on $\widetilde{X}$ representing smooth, irreducible curves whose $H$-degree equals $e$. Thus, for one of these classes, say $\beta$, there is an infinite subset $Y'_{\beta}\subset Y'_e$ such that for every $y\in Y'_{\beta}$, the strict transform $\widetilde{Z}_y$ has class $\beta$. Since $Y'_{\beta}$ is infinite, it contains at least two distinct elements $y,y'$. The smooth, genus $0$ Cartier divisors $\widetilde{Z}_y$ and $\widetilde{Z}_{y'}$ are linearly equivalent, and they are pairwise disjoint. Thus the self-intersection number of $\widetilde{Z}_y$ on $\widetilde{X}$ equals $0$. Since $\widetilde{Z}_y$ is a smooth, genus $0$ Cartier divisor with self-intersection $0$, the complete linear system $\Pi$ is a pencil: if the projective dimension is $\geq 2$, then for every $q\in \widetilde{Z}_y$, there is a pencil in this linear system of divisors that contain $q$, contradicting that the self-intersection equals $0$. QED
Notation 6. Denote by $\mathcal{Z}\subset \Pi\times \widetilde{X}$ the universal family of Cartier divisors. Denote by $\sigma_0:\Pi\to \mathcal{Z}$ the unique section of $\text{pr}_1$ such that $\nu\circ \sigma_0$ is a constant morphism to $p_0$. Denote by $f:\mathcal{Z}\to X$ the composition, $$\mathcal{Z}\xrightarrow{\text{pr}_2} \widetilde{X}\xrightarrow{\nu} X.$$ Denote by $R\subset \mathcal{Z}$ the non-injective locus, i.e., the image under either projection of the complement of the diagonal in $\mathcal{Z}\times_{X}\mathcal{Z}$. Denote by $R'$ the closure of $R\setminus \sigma_0(\Pi)$.
Proposition 7. There exists a dense open subset $\Pi^o\subset \Pi$ with inverse image $\mathcal{Z}^o=\mathcal{Z}\times_{\Pi}\Pi^o$ such that $f^o:\mathcal{Z}^o\setminus \sigma_0(\Pi^0) \to X$ is an open immersion.
Proof. By hypothesis, for all $y$ in the infinite set $Y'_\beta$, the curve $\widetilde{Z}_y$ is a fiber of $\text{pr}_1$ that is disjoint from $R'$. Thus, $R'$ is a union of finitely many irreducible components of fibers of $\text{pr}_1$. So there exists a dense open subset $\Pi^o\subset \Pi$ such that $R'$ is disjoint from the inverse image $\mathcal{Z}^o := \mathcal{Z}\times_{\Pi}\Pi^o$. Up to shrinking $\Pi^o$, we can also assume that the projection, $$\rho^o:\mathcal{Z}^o\to \Pi^o,$$ is a smooth morphism with geometric fibers isomorphic to $\mathbb{P}^1$. Finally, the section $\sigma_0$ restricts to a section of $\rho^o$. Thus, $\mathcal{Z}^o$ is a projective space bundle over $\Pi^o$ and the complement $\mathcal{Z}^o\setminus \sigma_0(\Pi^o)$ is an affine space bundle over $\Pi^o$.
By construction, this affine space bundle is disjoint from $R'$, and thus it is disjoint from $R$ (since we removed the image of $\sigma_0$). Thus $f^o$ is injective. Also $f^o$ is birational. Since $X$ is normal, by Zariski's Main Theorem, the morphism $f^o$ is an open immersion. QED
Corollary 8. For all but finitely many $y\in Y$, the curve $Z_y$ is the closure in $X$ of the image under $f^o$ of a fiber of $\rho^o$.
Proof. The complement of the image of $f^o$ is a closed subset of $X$ that has only finitely many irreducible components. Assume that $Z_y$ is not any of these irreducible components. Then the $f^o$-inverse image of $Z_y$ is an irreducible curve in $\mathcal{Z}^o$. Consider the projection of this curve to the irreducible curve $\Pi^o$. If this projection is not constant, then the image is an open subset whose complement consists of finitely many points. Since $Y'_\beta$ is infinite, there exist infinitely many $y'\in Y'_\beta$ such that $Z_y$ intersects the fiber $\widetilde{Z}_{y'}\setminus \{\sigma_0(y')\}$. Thus, $Z_y$ and $Z_{y'}$ intersect in a point different from $p_0$, contrary to hypothesis. It follows that $Z_y\setminus\{p_0\}$ equals one of the affine space fibers of $\rho^o$. QED
Proof of the Main Result. There is an induced open immersion $\Pi^o\to Y$ matching $Z_y$ with the fiber of $\rho^o$. This defines a rational transformation of $Y$ to the Chow variety of $1$-cycles on $X$ of $H$-degree $e$, and this rational transformation extends to a regular morphism on all of $Y$ by properness of the Chow variety. For the associated family of $1$-cycles, say $\mathcal{C}\subset Y\times X$, the induced morphism from the normalization $\mathcal{C}^{\text{nor}}$ to $\widetilde{X}_1$ is a finite, birational morphism of normal varieties, hence it is an isomorphism. In particular, since each of the finitely many curves $Z_y$ in the complement of $f^o(\mathcal{Z}^o)$ contains $p_0$, yet only one irreducible component of every fiber of $\mathcal{C}^{\text{nor}}\to Y$ intersects $p_0$, it follows that the fibers are irreducible. Finally, the additional hypothesis implies that each of these (scheme-theoretic) fibers is reduced, or else the $H$-divisor of the reduced fiber $Z_y$ properly divides $e$. Since every fiber is integral and smooth, isomorphic to $\mathbb{P}^1$, the scheme $\mathcal{C}^{\text{nor}}$ is a projective space bundle over $Y$.
Finally, the induced morphism $\mathcal{C}^{\text{nor}}\to X$ factors through the blowing down of the unique cross-section $\sigma_0:Y\to \mathcal{C}^{\text{nor}}$ having negative self-intersection. By the classification of rank $2$ locally free sheaves on $\mathbb{P}^1\cong Y$, this blowing down is isomorphic to a projective cone over a rational normal curve. The induced morphism from this cone to $X$ is finite and birational. Since $X$ is normal, by Zariski's Main Theorem, this induced morphism is an isomorphism. QED
Weighted Projective Planes. Without the additional hypothesis, there are many additional examples. Let $a_0,a_1,a_2\geq 1$ be integers that are pairwise relatively prime. Let $X$ be the weighted projective plane, $$\mathbb{P}(a_0,a_1,a_2) = \text{Proj}\ k[x_0,x_1,x_2], \ \ \text{deg}(x_i)=a_i.$$ Let $Y$ be the zero locus of $x_0$. For every $y=[0,y_1,y_2]$ in $Y$ with $y_1$ and $y_2$ nonzero, define $Z_y$ to be the zero locus of $y_2^{a_1}x_1^{a_2}-y_1^{a_2}x_2^{a_1}$. For $y=[0,0,1]$, resp. for $y=[0,1,0]$, define $Z_y$ to be the zero locus of $x_1$, resp. the zero locus of $x_2$.
All of the OP's hypotheses are true for these weighted projective planes, but the additional hypothesis is false unless $a_1=a_2$. The integer $e$ equals $a_1a_2e'$ for some integer $e'$, yet the two special curves $Z_y$ above have degrees $a_1e'$ and $a_2e'$, which are proper divisors of $e$.
When $a_1$ does equal $a_2$, then both equal $1$. In this case, the weighted projective plane is $\mathbb{P}(a_0,1,1)$. This is a cone over a rational normal curve of degree $d=a_0$. Thus, it may be that without the additional hypothesis, every example is a weighted projective plane.
