The exercise is correct.  Let $u\in B\setminus A$.  We say that $u$ splits an ultrafilter $H$ of $A$ if $\{u\}\cup H$ and $\{-u\}\cup H$ both have the finite intersection property.  (If $u$ splits $H$, then there are ultrafilters $F$ and $G$ of $B$ such that 
$F\cap A=H=G\cap A$, $u\in F$, and $-u\in G$.)

Suppose no ultrafilter $H$ is split by $u$.  
We say that an ultrafilter $H$ of $A$ is compatible with $b\in B$ iff each $a\in H$ has 
nonempty intersection (in $B$) with $b$.  If no ultrafilter of $A$ is split by $u$,
then each ultrafilter is either compatible with $u$ or compatible with $-u$.

Let $C$ be the set of ultrafilters of $A$ compatible with $u$, and let $D$ be the set of ultrafilters of $A$ compatible with $-u$.  
Now the set Ult$(A)$ of all ultrafilters of $A$ is the disjoint union of $C$ and $D$.
It is easily checked that the sets $C$ and $D$ are both open subsets of the Stone space Ult$(A)$ of $A$.  It follows that the two sets are clopen.
By the Stone representation theorem, there is $a\in A$ such that $C$ is the set of all ultrafilters $H$ of $A$ that contain $a$.  $D$ is the set of all ultrafilters of $A$ that contain $-a$.  

In other words, an ultrafilter $H$ of $A$ is compatible with $u$ iff $a\in H$.
But this implies that an element $b$ of $A$ has a nonempty intersection with $u$
iff it has nonempty intersection with $a$.  
This shows that $a\leq u$.  
The symmetric argument shows that $-a\leq-u$.
It follows that $a=u$ and hence $u\in A$, a contradiction.

And yes, this exercise implies that every infinite Boolean algebra of size $\kappa$ has at least $\kappa$ ultrafilters.