Ok, I think one can quickly reduce to the case where $A$ is local and also that $B$ is $A$-free of finite rank (I need to think about this latter assumption a little, but probably by completion we can do this).  

But granting these assumptions, we can proceed as follows (I think):

Set $A \to A'$ to be the absolute Frobenius on $A$ and likewise $B \to B'$ the Frobenius on $B$.  The primes will just help me distinguish source and target.

Then we want to show that the obvious map:
$$\Psi : A' \otimes_A B \to B'$$
is an isomorphism.

Now, $B'$ is a free $A'$ module, likewise so is $A' \otimes_A B$.  So we are trying to establish that a certain map of free $A'$-modules is an isomorphism.  First we show surjectivity, so we mod out by the maximal ideal $m' \subseteq A'$.  Then we obtain:
$$\Phi : A'/m' \otimes_{A/m} B/m \cong A'/m' \otimes_A B \to B'/m'B'.$$
Now, $A/m$ is a field, and $B/mB$ is a separable extension (by the etale hypothesis).  $A'/m'$ is a purely inseparable extension and so we easily see (by using that the extensions are linearly disjoint) that $B'/m'B'$ is identified naturally with $A'/m' \otimes_{A/m} B/m$.  

It follows that $\Phi$ is an isomorphism.  Thus $A' \otimes_A B \to B$ is surjective by Nakayama's Lemma.  But $A' \otimes_A B$ and $B'$ have the same rank as $A'$-modules by the above argument.  It follows that then $\Psi$ is a surjection between finite free modules of the same rank.  Thus $\Psi$ is an isomorphism.