No. Let $Y$ be $\Delta^1$ and let $X$ be the boundary of $\Delta^2$. Map $X$ to $Y$ by the simplicial map taking vertices $0,1,2$ to $0,1,1$.
In effect, by just looking at fibers over vertices you are not getting a grip on fibers over interior points of simplices.