Let $X\to \Delta$ be a projective family, smooth over $\Delta^*$, such that all fibers over $t\in \Delta^*$ are isomorphic. Does the monodromy representation factor through the algebraic automorphism group of the smooth fiber, $Aut(X_t)$?

This is certainly false for non-isotrivial families, since Dehn twists on curves are infinite order, whereas smooth curves of genus $>1$ have finite automorphism groups.

Example 1: The family $y^2 = x^3+t$ has monodromy of order 6, which is precisely the automorphism group of the Eisenstein elliptic curve.

Example 2: The family of smooth quadric surfaces $\mathbb P^1\times \mathbb P^1$ degenerating to a quadric cone has monodromy of order 2, which corresponds to swapping the factors.

In the symplectic category, there is a notion of parallel transport, so we have a monodromy map $\pi_1(\Delta^*)\to Symp(X_t)/Ham(X_t)$. Perhaps if the family is algebraically isotrivial, then the monodromy is valued in $Aut(X_t)$ and is homotopy invariant, so there is no quotient?