Yes, this is true. Suppose that $T$ is the given tangle in the three-ball, and the branched double cover is a graph manifold. By the uniqueness of the JSJ decomposition, the decomposition of the graph manifold into Seifert manifolds descends to give a tangle decomposition of $T$. This reduces full case to your simpler case where the branched double cover is a Seifert fibered space with boundary.
Applying an equivariant torus theorem, we can further reduce to the case where the branched double cover is Seifert fibered over the disk with two cone points. That is, to torus knot complements, where the result holds.