As pointed out above, the answer of Count Dracula is also valid, mutatis mutandis, if one desires counter-examples where the field extensions are of characteristic $p>2$. It turns out that it can also be adapted to provide an example where the characteristic of the base field extension is $0$, and is $2$ for the residual field extension. Here is how:

Let $R={\mathbb Z}[t_i, x_i, i\in \mathbb N]$, and $L$ the fraction field of $R$. Define the order 2 automorphism $\sigma$ of $R$ by $\sigma(z\in \mathbb Z)=z$, $\sigma(t_i) = t_i$ and $\sigma(x_i) = t_i-x_i$.

Let $S$ be the fixed sub-ring under the action, and $K$ its fraction field.
Then $\sigma$ extends to a $K$-automorphism of $L$ in an obvious manner, hence $[L:K]=2$ by Galois theory.
$R$ is integrally closed (since factorial), and is integral over $S$ because every $f\in R$ satisfies an integral dependance relation over $S$ of the form $(X-f)(X-\sigma f)=0$. It follows easily that $S$ must be integrally closed too.

We consider the ideals ${\frak P} =(2,t_i)\subset R$, and ${\frak p} = S\cap {\frak P}\subset S$. It is not difficult to see that $\frak P$, and hence $\frak p$, is prime. Now, as in the proof of Count Dracula, we see that $f\in R$ can be written
$$f_0 + \sum_i t_i f_i + \hbox{terms multiples of $t_j t_k$},$$
where $f_0, f_i$ depends only on the $x_j$ (eventually, consider the polynomial $f_0$ lacunary in some variables, and add some null $f_i$ in order for the variables in all the $f_0,f_i$ to be the same). If $f\in S$, then
$$ \sigma f(\star) = f_0(-\star) + \sum_i t_i(f_i(-\star) + {\partial f_0/\partial x_i}(-\star)) + \hbox{terms multiples of $t_j t_k$}.$$

Therefore $$\sum_i t_i \partial f_0/\partial x_i(-\star) = f_0(\star)-f_0(-\star) + \sum_i t_i(f_i(\star) - f_i(-\star)).$$ But $-1 = 1 \pmod2$ hence $ \partial f_0/\partial x_i = 0 \pmod2$. It follows, as in the proof of Count Dracula, that $A/{\frak p}={\mathbb F_2}(x_i^2)$, and that $B/{\frak P}={\mathbb F_2}(x_i)$ is infinite over $A/\frak p$. Considering finitely many variables, it can be shown similarly that $[L:K]=2$ is not divisible by $[B/{\frak P}:A/\frak p]$.

NOTE: to obtain a more general counter-example, it suffices to consider the order $p$ automorphism
$$\sigma(t_i)=t_i,\quad \sigma(z\in {\mathbb Z}[\zeta])=z,\quad \sigma(x_i)= t_i+\zeta x_i,$$
where $\zeta$ is a p-root of unity. The adaptations are straightforward, except, perhaps, the fact that the ideal $(2,t_i)$ must be replaced by the ideal ${\frak P}= (1-\zeta, t_i)$ and not $(p,t_i)$: this is because $(1-\zeta)$ is the only maximal ideal above $(p)$ in ${\mathbb Z}[\zeta]$.