Over $\mathbb Z$, there are several possibilities.
- If both $A, B$ are torsion-free over $\mathbb Z$, then $A\otimes_{\mathbb Z} B$ is connected if and only if its generic fiber $(A\otimes_{\mathbb Z} \mathbb Q)\otimes_{\mathbb Q} (B\otimes_{\mathbb Z} \mathbb Q)$ is connected, and you are reduced to the case of algebras over a field.
EDIT the above claim is incorrect. See a partial answer herehere.
If $A$ (or $B$) have positive characteristic, then $A_{\rm red}$ is an algebra over a product of finite fields, then so is $A\otimes_{\mathbb Z} B$. Again you are reduced to the case of algebras over a field (if at least two finite fields involve really, then the tensor product is not connected.
The remainning case: if $A$ is noetherian, then ${\rm Spec}(A)$ has a flat part and a ''vertical part'': take the ideal $I$ of $A$ consisting in torsion elements. Then $A/I$ is flat and $mI=0$ for some non-zero integer $m$, and we have ${\rm Spec A}=V(I)\cup V(mA)$.
The same is true for $A\otimes B$. To have the connectedness, you want [EDIT: it is enough (but far to be necessary) that] the flat and vertical parts to be connected and meet each other. Sorry, I don't have a simple statement.