Suppose $F\subset V\subsetneq \mathbb {R}$ where $F$ is closed and $V$ is open. I want to show that $\exists$ an open set $U\subset \mathbb {R}$ satisfying $F\subset U\subset \overline {U}\subset V$. My proof is below, but I think it's problematic since I never use the condition that $F$ is closed.

For each $x\in F$, pick $r_{x}>0$ such that $(x-r_{x},x+r_{x})\subset V$. Then define $U=\cup\{(x-\frac {r_{x}}{2},x+\frac {r_{x}}{2})\}\subset V$. Then in fact, $F\subset U$. This is because countable union of open sets is open and $U$ is an open cover of $F$. It's trivial that $U\subset \overline{U}$. Now, what remains is to show that $\overline{U}\subset V$. First of all, $U\subset V$ since $U$ is just a collection of open intervals, and each one of them is in $V$. In fact, under this construction, $U\subsetneq V$. Therefore, for $y\in \overline{U}\setminus U$, $\exists u\in U$ such that $y\in B(u,r_{u})$. Take $r_{y}=r_{u}-d(y,u)$. Then $B(y,r_{y})$ is an open ball inside of $V$. Thus, $y\in V$ and $\overline{U}\subset V$.