Let $A\subseteq B$ be commutative Noetherian rings such that $A$ is a regular ring, i.e., $A_{\mathfrak{m}}$ is a regular local ring for all maximal ideals $\mathfrak{m}$ of $A$ and $B$ is a finite $A$ module. If $\mathfrak{p}$ varies over the set of prime ideals of $A$, does the vector spacedimension of $B\otimes_A\kappa(\mathfrak{p})$ over $\kappa(\mathfrak{p})$ remains constant? What happens if we allow $\mathfrak{p}$ only to be maximal ideals? What if $A$ if further an affine domain over a field $k$ (which is algebraically closed or/and of characteristic $0$)?

2$\begingroup$ What happens if $B$ is a quotient of $A$? $\endgroup$ – Daniel Litt May 25 '16 at 5:13
That's false. In general the dimension is only upper semicontinuous. The simplest counterexample which comes to my mind is $A=\mathbb C[x^2,x^3]\subseteq B=\mathbb C[x]$. Then the dimension over the maximal ideal $(x^2,x^3)$ is $2$. All other dimensions are $1$ since $A$ and $B$ become equal after inverting $x^2$. For your statement to become true you need that $B$ is a flat $A$module. Equivalently $B$ should be locally free or projective as an $A$module. Addendum: That's automatic if $A$ is regular and $B$ CohenMacaulay (e.g. also regular).
Answer to the edit: If $A$ is regular then $B$ being locally free over $A$ is pretty much equivalent to $B$ being CohenMacaulay. So every Noether normalization of a nonCM algebra will be a counterexample. Maybe the simplest example is as follows: Let $\mathfrak m:=(x,y)\subseteq\mathbb C[x,y]$ and $B:=\mathbb C+\mathfrak m^2$. Take $A:=\mathbb C[x^2,y^2]$. Then the dimension over the maximal ideal $(x^2,y^2)$ is $6$ but $4$ everywhere else.