How about we try the simple case $R=S$. R=S$ and the identity map. Let $\mathfrak q$ be a singular prime. The fibre over $\mathfrak q$ is $\kappa (\mathfrak q)$ which is regular (it is a field) but $R_q$ R_{\mathfrak q}$ is not regular.
Edit: let me add a few more details since my original terse answer hardly deserves the upvotes! Let $P$ be the set of primes in $\text{Spec}(R)$ which contracts to $\mathfrak q $ (the set-theoretic fibre). Your first condition says that:
(1) $R_{\mathfrak p}$ is regular for all $\mathfrak p \in P$.
while the second says:
(2) $R_{\mathfrak p}/\mathfrak qR_{\mathfrak p}$ is regular for all $\mathfrak p \in P$.
So you can see where the problems come from: in general, for a local ring $R$, regularity of $R$ has nothing to do with regularity of $R/I$ for some ideal $I$, unless if $I$ is very special. That is how I think about the counter-examples, hopefully it helps.
