Let me explain my comment in a bit more detail.

**Proposition:**  *Suppose $B \subseteq A$ is an integral extension of domains and $P$ is a prime of $B$ with $Q \subseteq A$ a prime of $A$ lying over $B$.  Then $B_P \subseteq A_Q$ is integral if and only if $Q$ is the unique prime of $A$ lying over $B$.*

*Proof:*  Suppose first that $Q$ is the unique prime lying over $P$ and let $W = B \setminus P$.  Then $B_P = W^{-1} B \subseteq W^{-1} A$ is an integral extension (via an easy computation -- clear denominators as appropriate, find the integral relation, and then put the denominators back).  

**Claim:** $W^{-1} A \cong A_Q$.  

*Proof of claim:* Obviously $W \subset A \setminus Q$ and equality won't help, but we can still have the claim.  Indeed, suppose that $Q_1$ is a prime of $A$ corresponding to a prime of $W^{-1} A$.  We will show that $Q_1$ is contained in $Q$.  Let $P_1 = B \cap Q_1$.  If $P_1 \subseteq P$, then by the going up theorem, there exists $Q'$ over $P$ with $Q_1 \subseteq Q'$.  But $Q' = Q$ by the uniqueness hypothesis which proves that $Q_1$ is contained in $Q$ as desired.  This shows that $W^{-1} A$ is a local ring with maximal ideal $W^{-1} P$.  But then by the universal property of localization, $W^{-1} A \cong A_P$ which proves the claim.

The claim obviously shows that $B_P \subseteq A_Q$ is integral.   The above direction doesn't need the domain hypothesis.

For the converse direction, suppose that $Q_1, Q_2$ are distinct primes of $A$ lying over $P$.  If $W = B \setminus P$ and $B_P = W^{-1} B \subseteq A_{Q_1}$ is integral, then certainly $W^{-1} A \subseteq A_{Q_1}$ is also integral (we are just enlarging our ring of coefficients).  Note that if $A$ is not a domain, it doesn't necessarily follow that $W^{-1} A \subseteq A_{Q_1}$ (this is where the domain hypothesis is used).  But now obviously we have $(W^{-1} Q_2) A_{Q_1} = Q_2 A_{Q_1} = A_{Q_1}$ and so there is no prime of $A_{Q_1}$ lying over $W^{-1} Q_2$, a contradiction.  This proves the proposition.


**Remark:** In the non-domain case, the proposition can fail.  Consider $k \subseteq k \oplus k$ via the diagonal inclusion.  There are two primes of the overring lying over $\langle 0_k\rangle$, and localizing $k \oplus k$ at either just yields $k$ again.  However, as noted in the proof above, generalizations of this phenomonon are essentially the only way it can fail for non-domains.