The exercise is correct. Let $u\in B\setminus A$. We say that $u$ splits an ultrafilter $H$ of $A$ if $\{u\}\cup H$ and $\{-u\}\cup H$ both have the finite intersection property. (If $u$ splits $H$, then there are ultrafilters $F$ and $G$ of $B$ such that $F\cap A=H=G\cap A$, $u\in F$, and $-u\in G$.)
Suppose no ultrafilter $H$ is split by $u$.
We say that an ultrafilter $H$ of $A$ is compatible with $b\in B$ iff each $a\in H$ has
nonempty intersection (in $B$) with $b$. If no ultrafilter of $A$ is split by $u$,
then each ultrafilter is either compatible with $u$ or compatible with $-u$.
Let $C$ be the set of ultrafilters of $A$ compatible with $u$, and let $D$ be the set of ultrafilters of $A$ compatible with $-u$.
Now the set Ult$(A)$ of all ultrafilters of $A$ is the disjoint union of $C$ and $D$.
Hence an ultrafilter of $A$ is compatible with $u$ iff it is not compatible with $-u$ and vice versa. So, if $H\in C$, then there is $a\in H$ such that $a$ is disjoint from $-u$.
All ultrafilters of $A$ that contain $a$ are incompatible with $-u$ and hence compatible with
$u$. This shows that $C$ is open in the Stone space Ult$(A)$ of $A$.
The same is true for $D$. It follows that the two sets are clopen.
By the Stone representation theorem, there is $a\in A$ such that $C$ is the set of all ultrafilters $H$ of $A$ that contain $a$. $D$ is the set of all ultrafilters of $A$ that contain $-a$.
In other words, an ultrafilter $H$ of $A$ is compatible with $u$ iff $a\in H$. But this implies that an element $b$ of $A$ has a nonempty intersection with $u$ iff it has a nonempty intersection with $a$. Hence $-a$ is disjoint from $u$. In other words, $u\leq a$. The symmetric argument shows that $-u\leq-a$. It follows that $a=u$ and hence $u\in A$, a contradiction.
And yes, this exercise implies that every infinite Boolean algebra of size $\kappa$ has at least $\kappa$ ultrafilters.