=========
Thank Qing Liu for the reference. Today I thought I had a way to prove this. Suppose $\phi : A \rightarrow B$ has degree $p^r$. Let $\psi := \times p^r : A \rightarrow A $. In order to show that the existence of $ \hat{ \phi} $, one only need to show that $\psi^* (K(A))$ is contained in $\phi^* (K(B))$, here $K(A)$ and $K(B)$ are the function fields of $A$ and $B$. (Then we have a rational map from $B$ to $A$ and extends to a morphism $ \hat{ \phi }$.) But notice that for each element $g \in K(A)$, $g^{p^r} \in \phi^* (K(B))$. If one can show that for each $f \in K(A)$, there exists $g \in K(A)$ such that $\psi^* (f) = g^{p^r}$, then we are done.
Since the multiplication by $p$ morphism $ [p] : A \rightarrow A$ is the composition of the Frobenius morphism $F: A \rightarrow A^{(p)}$ with the Verschiebung morphism $V : A^{(p)} \rightarrow A$, one sees that for any $f \in K(A)$, $[p]^* (f) = g^p$ for some $g \in K(A)$, if $k$ is perfect. In particular, this holds for an algebraically closed field $k$. So in this case, the statement in the end of the previous paragraph is true.
Finally, for $k$ not necessary being algebraically closed, we take a base change and work on $\overline{k}$ first. We then get $\overline{\psi} = \Phi \circ \overline{\phi}$, here $\Phi : \overline{B} \rightarrow \overline{A}$ and the $ \ \overline{ \ }$ means the schemes and the morphisms obtained from the base change. For any $f \in K(A)$, if one can show that $\Phi^* (f) \in \phi^* (K(B))$, then one completes the proof. But we have $\Phi^* (f) \in K(A) \cap \overline{\phi}^* (\overline{K}(B)) = K(A) \cap \phi^* (K(B)) \otimes_k \overline{k}$. Consider the subfield extension $\phi^* (K(B)) (\Phi^* (f))$ of $K(A)/\phi^* (K(B))$, then $\Phi^* (f) \in K(A) \cap \overline{\phi}^* (\overline{K}(B))$ gives $\phi^* (K(B)) (\Phi^* (f)) = \phi^* (K(B))$, and we are done.
Just a remark about the factorization of the morphism $[p] = V \circ F$. This is somehow different with our $\phi$ and $\hat{ \phi }$, since $\mathrm{deg}(F) = p^g$, not $p$, so it's a stronger result. Also one can show that the kernel of $F$ is of this form (via a $k$-isomorphism) $k[X_1, \cdots, X_n]/(X_1^p, \cdots, X_n^p)$. For a commutative group scheme of this form, I think one can show that the morphism $[p]$ on it is the zero morphism $0$ by dealing with symmetric functions. (I am not sure about this, but I saw somewhere that the Verschiebung morphism $V$ also uses the symmetric functions.) I like this way of approach since it uses the description of $[p]$ by the Frobenius morphism $F$, and the kernel $K$ of $F$ is explicit (if one chooses a coordinate). However, in Mumford's book, there is a proof that $[p] = 0$ for height one commutative group scheme by using the Lie-algebra.
