The first part of the answer is an extension of the comment by R. van Dobben de Bruyn.
We have
$$\textit{ the class of open immersions of schemes } = \textit{ the class of étale monomorphism of schemes} $$
Hence we deduce (by Zariski's main theorem) that $\phi^*:A[Y] \rightarrow A[X]$ is quasi-finite if and only if there exists an affine $k$-scheme $Z$ and a factorization $\phi^* = i^*\cdot p^*$ such that $p^*:A[Y] \rightarrow A[Z]$ is a finite morphism and $i^*:A[Z] \rightarrow A[X]$ is an étale epimorphism of $k$-algebras. Now étale can be characterized as formally étale and (in case of finitely generated $k$-algebras) of finite type. For characterizing epimorphisms of rings you may be interested in the following MO question.
For your second question it suffices to assume (in addition to $\mathrm{dim}(X) = \mathrm{dim}(Y)$) that $\phi:X\rightarrow Y$ is flat. By flatness all fibers have the same dimension. Moreover, flat morphism locally of finite presentation are open, thus $\phi$ is dominant and hence the generic fiber of $\phi$ is finite. Hence all fibers are finite and $\phi$ is quasi-finite (clearly it is of finite type).
Moreover, if $\phi:X\rightarrow Y$ is a dominant morphism of varieties such that $\mathrm{dim}(X) = \mathrm{dim}(Y)$, then there exists open dense subset $V\subseteq Y$ such that $\phi^{-1}(V)\rightarrow V$ is finite, but in general not all fibers are finite as it was pointed by R. van Dobben de Bruyn in the comments.