No. Let $Y$ be $\Delta^1$ and let $X$ be the boundary of $\Delta^2$. Map $X$ to $Y$ by the simplicial map taking vertices $0,1,2$ to $0,1,1$.
In effect, by just looking at fibers over vertices you are not getting a grip on fibers over interior points of simplices.
(Edited later: This is wrong: I overlooked the requirement that $X$ and $Y$ should be Kan complexes. But the examples given by others in the comments make the same point without being wrong.)