Here is a counterexample. It's roughly based on the following useful example: if $X$ is the amount you decide to bet on a fair game, and $Y$ is your resulting profit, then $X,Y$ are uncorrelated but not independent.
Let $\xi_n$ be independent with $P(\xi_n = n) = n^{-2}$, $P(\xi_n = 0) = 1-n^{-2}$. Note that $E[\xi_n] = n^{-1}$ and that, by Borel–Cantelli, we have $\sum_n \xi_n < \infty$ a.s.
Let $\theta$ be a fair coin flip (Bernoulli 1/2) independent of all the $\xi_n$. Let $\mathcal{F}_n = \sigma(\theta, \xi_1, \dots, \xi_n)$.
Set $X_n = n^{-1} \theta + \xi_n (1-\theta)$, which is $\mathcal{F}_n$-measurable. Notice that $$E[X_n \mid \theta = 1] = n^{-1} = E[\xi_n] = E[X_n \mid \theta = 0] $$ and that $(X_n, \theta)$ is independent of $(\xi_1, \dots, \xi_{n-1})$. Hence $E[X_n \mid \mathcal{F}_{n-1}] = E[X_n \mid \theta] = n^{-1} = E[X_n]$, so we have the desired condition.
But on the event $\theta = 1$ we have $X_n = n^{-1}$ for all $n$, and then $\sum X_n = \infty$. And if $\theta = 0$ then $X_n = \xi_n$ for all $n$, then $\sum X_n < \infty$. So we have $P(B) = 1/2$.
(Incidentally, the sequence $M_n = \sum_{k=1}^n (X_k - k^{-1})$ is a martingale which converges with probability $1/2$. In fact, in retrospect, I think the desired condition is equivalent to "$X_n$ is a martingale difference sequence plus constants.")
If you require the $X_n$ to be uniformly bounded then the answer is yes; $P(B)$ is 0 or 1 according to whether $\sum_n E[X_n]$ converges or diverges.
Let's say $0 \le X_n \le 1$ for all $n$. Let $c_n = E[X_n]$. If $\sum_n c_n < \infty$ then by monotone convergence we have $E\left[\sum_n X_n\right] < \infty$ and thus $P(B) = 0$. So suppose that $\sum_n c_n = \infty$. Let $Y_n = X_n - c_n$ (so $-1 \le Y_n \le 1$) and $M_n = \sum_{k=1}^n Y_k$. As noted above, $M_n$ is a martingale, which now has bounded increments. On the event $B^c$ we have $M_n = \sum_{k=1}^n X_k - \sum_{k=1}^n c_k \to -\infty$ since the first sum converges and the second diverges. However, according to this exercise, there is probability zero to have $M_n \to -\infty$. (It says that almost surely, $\limsup M_n$ is either finite or $+\infty$.) So in this case $P(B)=1$.