[Nitpick: I think Urysohn's lemma concerns existence of functions which are 1 and 0 on two given closed subsets; the property here, of being supported inside an open neighbourhood and non-zero, is noticeably weaker. The property described/needed here is more like being a "regular" function algebra. Compare the difference between a normal topological space and a completely regular topological space]
I will formulate everything for general locally compact groups, even though at the place where Folland needs this result, he is assuming all groups are locally compact abelian.
Let me first address the case where $x=e$ and $U$ is an open neighbourhood of $e$. By general topology, since $G$ is locally compact we may assume that there is an open neighbourhood $V\ni e$ whose closure is compact and contained in $U$. Moreover, we can find a compact neighbourhood $K\ni e$ in $G$ such that $KK^{-1}:=\{ tu^{-1} \mid t,u\in K\} \subseteq V$.
Since $K$ has non-empty interior it has strictly positive Haar measure. Writing $|K|$ for the (Haar) measure of $K$, we now define $\xi = |K|^{-1/2} 1_K$ and form the coefficient function
$$
f(s) = \langle \lambda_s \xi, \xi\rangle = |K|^{-1} \int 1_K(s^{-1}t) 1_K(t)\,dt =|K|^{-1} |K\cap sK|
$$
$f$ is a positive definite function (in the language used in Folland's book, it is a function of positive type). In particular $f\in {\mathcal B}(G)$.
Moreover, $f$ is supported inside $V$: for if $K\cap sK\neq\emptyset $, let $t\in K\cap sK$ and put $y=s^{-1}t\in K$, yielding $s=ty^{-1}\in K K^{-1} \subseteq V$.
Since $V$ has compact closure, $f\in C_c(G)\subseteq L^1(G)$; and since $f$ vanishes outside $V$ it certainly vanishes outside $U$, as originally required.
The general case, as stated in the OP's question, now follows from the case $x=e$ that was done above, and the fact that ${\mathcal B}^1(G)$ is invariant under left translations. Alternatively one can run the argument above but with an appropriate left translation inserted everywhere: the resulting $f$ will still be a coefficient function of the left regular representation, but it won't usually be a function of positive type.
Remarks.
- If you haven't seen this construction of small "triangular" bump functions it is instructive to do this for, e.g. $G={\mathbb T}$ and $x=1$, $U= \{ e^{2\pi it} \mid -\delta< t <\delta\}$ for some small positive $\delta$.
- I am not quite sure why Folland refers to Proposition 3.33, except that it says ${\mathcal B}(G)$ does contain $f*g$ for any $f,g\in C_c(G)$. Initially I thought he might be using one of the density statements that are mentioned in the statement of the proposition, but this doesn't seem to play a role.