It's too late for me to be careful with the details on this, but let me put this down to think about later. I don't immediately see an error, but it goes against my gut instinct that the answer "should"
This should be yesa counterexample.
Let $n$ be sufficiently large, and let $Y = \mathbb{G}(2,n)$ be the Grassmannian of $\mathbb{P}^2$'s in $\mathbb{P}^n$. Take $X = \{ (p,\Lambda):p\in\Lambda \}$ to be the universal $2$-plane over $Y$. Let $S\subset \mathbb{P}^n$ be a surface which contains a line $L$ (perhaps a rational surface scroll, but there are lots of things to try), and let $Z = \{(p,\Lambda): p\in S \}\subset X.$ Then $Z$ is irreducible. I'd expect that the general $2$-plane $\Lambda_0$ which contains $L$ will not intersect $S$ in any other points (say $n\geq 5$), i.e. that the corresponding fiber $Z_{\Lambda_0} \subset X_{\Lambda_0} \cong \mathbb{P}^2$ is a Cartier divisor. But for more special choices of $2$-plane containing $L$, the intersection will be $L$ together with a finite collection of points, and so the fiber will not be a divisor.
