If $|A|=1$ and $|B|$ is very large, then with extremely high probability, someone in $B$ will receive a net transfer of 1.  However, the net transfer to the person in $A$ is negative as often as it's positive, and if the cardinality of $B$ is even, it's also sometimes zero.  So I think your conjectured result cannot be true in this case.