The exercise is correct. Let $u\in B\setminus A$. We say that $u$ splits an ultrafilter $H$ of $A$ if $\{u\}\cup H$ and $\{-u\}\cup H$ both have the finite intersection property. (If $u$ splits $H$, then there are ultrafilters $F$ and $G$ of $B$ such that $F\cap A=H=G\cap A$, $u\in F$, and $-u\in G$.)
Suppose no ultrafilter $H$ is split by $u$.
We say that an ultrafilter $H$ of $A$ is compatible with $b\in B$ iff each $a\in H$ has
nonempty intersection (in $B$) with $b$. If no ultrafilter of $A$ is split by $u$,
then each ultrafilter is either compatible with $u$ or compatible with $-u$.
Let $C$ be the set of ultrafilters of $A$ compatible with $u$, and let $D$ be the set of ultrafilters of $A$ compatible with $-u$.
Now the set Ult$(A)$ of all ultrafilters of $A$ is the disjoint union of $C$ and $D$.
It is easily checked that the sets $C$ and $D$ are both open subsets of the Stone space Ult$(A)$ of $A$. It follows that the two sets are clopen.
By the Stone representation theorem, there is $a\in A$ such that $C$ is the set of all ultrafilters $H$ of $A$ that contain $a$. $D$ is the set of all ultrafilters of $A$ that contain $-a$.
In other words, an ultrafilter $H$ of $A$ is compatible with $u$ iff $a\in H$.
But this implies that an element $b$ of $A$ has a nonempty intersection with $u$
iff it has nonempty intersection with $a$.
This shows that $a\leq u$.
The symmetric argument shows that $-a\leq-u$.
It follows that $a=u$ and hence $u\in A$, a contradiction.