Denote the curve class by $\beta$. Let $M$ be the normalization of a closed subvariety of $\overline{\mathcal{M}}_{0,2}(X,\beta)$ such that the restricted evaluation morphism, $$\text{ev}|_M = (\epsilon_1,\epsilon_2):M\to X\times X,$$ is surjective and generically finite. Denote by $m$ the degree of $\text{ev}|_M$. The claim is, for For every morphism $f:Y\to X$ as above, the claim is that $$\langle [D_f], \beta \rangle \geq 2d - 2(m!).$$
Associated to $f^o$, there is also a monodromy representation $\rho$ of $\pi_1(X^o,x_1)$ on $(f^o)^{-1}(x_1)$, and the action is again transitive since $Y$ is normal and connected. Consider the induced action of $N$ on $(f^o)^{-1}(x_1)$. Since $\rho$ is transitive, and since $N$ has index $\mu$, there are at most $\mu$ orbits for $\rho|_N$.
For every pair $y_2$$y'$, $y_3$$y''$ of points of $(f^o)^{-1}(x_1)$ that are in the same orbit of $N$, there is a real continuous path $\gamma$ in $Y^o$ from $y_2$$y'$ to $y_3$$y''$ whose image under $f^o$ is a loop $f(\gamma)$ in $X^o$ based at $x_1$ and whose homotopy class is in $N$. In particular, the loop lifts to a loop $\gamma_M$ in $M^o$ based at $m_1$.
As we vary the point $m_1$ in $M^o$, the point $x_0$ and the corresponding partition of $(f^o)^{-1}(x_0)$ do not vary. Thus, if we take athe loop $\gamma_M$ in $M^o$ based at $m_1$, and if we analytically continue $y'$ along this map according to the path $\gamma$, if $y'$ was in the same irreducible component of $\widetilde{C}$ as $y$ at the beginning of the analytic continuation, then the conjugate point $y''$ of $y'$ must be in the same irreducible component of $\widetilde{C}$ as $y$ at the end of the analytic continuation. Therefore, every pair of $N$-conjugate points of $(f^o)^{-1}(x_1)$ are in the same irreducible component of $\widetilde{C}$.
Finally, since there were at most $\mu$ distinct orbits of $N$ acting on $(f^o)^{-1}(x_1)$, it follows that $\widetilde{C}$ has at most $\mu$ distinct irreducible components. For each irreducible component $\widetilde{C}_i$, denoting by $d_i$ the degree of that component over $\text{Image}(u)$$\text{Image}(u_{m_1})$, denoting by $g_i$ the arithmetic genus of (the normalization of) that component, and denoting by $b_i$ the branch number of that component, then Riemann-Hurwitz gives
$$
2g_i-2 = d_i(-2) + b_i, \ \ b_i = 2d_i - 2 + 2g_i \geq 2d_i - 2.
$$
Summing up over all irreducible components, we have,
$$
\langle [D_f], \beta \rangle \geq 2d - 2\mu \geq 2d - 2(m!).
$$