Let $A$ be a commutative Noetherian ring and $B$ a finitely generated $A$-algebra. Then the set $$U\colon=\{P\in\operatorname{Spec}B\mid B_P\ \mathrm{is\ flat\ over}\ A\}$$ is open in $\operatorname{Spec}B$. (See, e.g., page 187 of Matsumara's Commutative Ring Theory.)
Is there a known example of a non finitely generated $A$-algebra $B$, where the set $U$ as defined above is not open?
Here is one example. Let $S$ denote the set of positive prime integers. Let $A$ be $\mathbb{Z}$. Let $R$ be the countably generated polynomial ring over $\mathbb{Z}$, $$R = \mathbb{Z}[\{x_p:p\in S\}].$$ Let $I\subset R$ be the ideal generated by $\{px_p : p\in S\}$. Let $B$ be $R/I$. Then the ideal $\mathfrak{p}$ of $B$ generated by $\{x_p : p\in S\}$ is prime, since $B/\mathfrak{p}$ is just the integral domain $\mathbb{Z}$. For every integer $p\in S$, $p$ is not in $\mathfrak{p}$. Hence, the localization of $B$ at $\mathfrak{p}$ factors through $B\otimes_{\mathbb{Z}}\mathbb{Q}$. But, of course, this localization is already $(B/\mathfrak{p})\otimes_{\mathbb{Z}}\mathbb{Q}$, which is $\mathbb{Q}$. Since $\mathbb{Q}$ is flat over $\mathbb{Z}$, thus $\mathfrak{p}$ is in $U$.
Every Zariski open subset of $\text{Spec}(B)$ containing $\mathfrak{p}$ contains a basic open subset of the form, $$D(b)=\{\mathfrak{q} \in \text{Spec}(B) : b\not\in \mathfrak{q}\},$$ for some $b\in B\setminus \mathfrak{p}$. Since $b$ is not in $\mathfrak{p}$, $b$ equals $n+c$ for some $c\in \mathfrak{p}$ and for some nonzero $n\in \mathbb{Z}$. Of course $c$ is a polynomial in only finitely many of the variables $x_p$. Also $n$ is divisible by only finitely many primes. Hence there exists a prime $q$ such that $c$ does not involve $x_q$, and, also, $q$ does not divide $n$.
Consider the unique surjective $\mathbb{Z}$-algebra homomorphism, $$ u_q : R \to (\mathbb{Z}/q\mathbb{Z})[x_q],\ \ u_q(x_q) = x_q, \ \ u_q(x_p) = 0,\ p\neq q. $$ Clearly $\text{Ker}(u_q)$ contains $px_p$ for every $p\neq q$, since $u_q(x_p)$ equals $0$. But since $q$ equals $0$ in $\mathbb{Z}/q\mathbb{Z}$, also $\text{Ker}(u_q)$ contains $qx_q$. Hence $u_q$ factors through a unique surjective ring homomorphism, $$ \overline{u}_q: B \to (\mathbb{Z}/q\mathbb{Z})[x_q].$$ Since $(\mathbb{Z}/q\mathbb{Z})[x_q]$ is an integral domain, the ideal $\mathfrak{q}:=\text{Ker}(\overline{u}_q)$ is a prime ideal of $B$. Also, since $\overline{u}_q(c)$ equals $0$, $\overline{u}_q(b)$ equals $\overline{u}_q(n)$. Since $q$ does not divide $n$, $\overline{u}_q(b)$ is nonzero. Thus $\mathfrak{q}$ is in $D(b)$.
Of course for every prime $p\neq q$, since $\overline{u}_q(p)$ is nonzero, also $p$ is not in $\mathfrak{q}$. Thus the localization $B\to B_{\mathfrak{q}}$ factors through $B\otimes_{\mathbb{Z}} \mathbb{Z}_{(q)}$, which is clearly just $\mathbb{Z}_{(q)}[x_q]/\langle qx_q \rangle$. Of course the image of $\mathfrak{q}$ in this localization is the principal ideal $\langle q \rangle$. In particular, $x_q$ is not in this prime ideal. Hence, the localization inverts $x_q$, and thus annihilates $q$. So $B_{\mathfrak{q}}$ is simply the field $(\mathbb{Z}/q\mathbb{Z})(x_q)$ of rational functions in the variable $x_q$ over the field $\mathbb{Z}/q\mathbb{Z}$.
The field $(\mathbb{Z}/q\mathbb{Z})(x_q)$ is not flat over $\mathbb{Z}$, since $q$ is a zerodivisor. Thus $\mathfrak{q}$ is not in $U$. Therefore $D(b)$ is not contained in $U$. Since this holds for every $b\in B\setminus \mathfrak{p}$, $U$ contains no Zariski open neighborhood of $\mathfrak{p}$, even though $U$ contains $\mathfrak{p}$. Therefore $U$ is not a Zariski open subset of $\text{Spec}(B)$.
Edit. In fact, it is not hard to see that $U$ is precisely $\{\mathfrak{p}\}$ for this ring.
Second Edit. I realize now that the ring $B$ above is "almost" a subring of the ring proposed by user76758 in the comments (I did not see that proposal until after I posted). Let $J$ be the ideal in $R$ generated by $px_p$ and $x_p^2-x_p$ for every $p$ in $S$. Then $C=R/J$ is still a counterexample, for essentially the same reason as above. Also $C$ is isomorphic to the $\mathbb{Z}$-subalgebra of $\prod_p \mathbb{F}_p$ generated by every element $\overline{x_p}$ that has coordinate $1$ in the $p$-factor and that has $0$ in every other factor.
Here's a slightly more "geometric" example.
Take a smooth affine $3$-fold $X = \mathrm{Spec}(R)$ over the complex numbers, and let $X^+ \to X$ be an absolute integral closure of $X$ in the sense of Artin, i.e., $X^+$ is the normalisation of $X$ in an algebraic closure of its function field. We may view $X^+$ as a directed inverse limit of finite (normal) covers $Y \to X$ along finite surjective transition maps. Let $U$ be the flat locus of $\pi:X^+ \to X$.
Claim: $U = \pi^{-1}(X - X^0)$, where $X^0 \subset X$ is the set of closed points.
Proof: To show $\supset$, note that a finite normal extension of a regular local ring of dimension $\leq 2$ is automatically flat. To show $\subset$, one must check that $\pi$ is not flat at any closed point of $X$. This can be shown using local cohomology (using crucially the characteristic $0$ assumption).
Finally, it remains to observe that $U$ is not open: if it were open, it would arise as the inverse image of an open $V \subset Y$ for some finite cover $Y \to X$ occurring in the inverse limit defining $X^+$. However, any such $V$ must contain a closed point of $Y$. As $X^+ \to Y$ is surjective, $U$ contains a point mapping to a closed point of $X$ (through $Y$), which cannot happen by the claim above.
