The situation is similar to the one of closed manifolds. One defines "boundary-prime" manifolds as those that cannot be decomposed nontrivially in a boundary-connected sum.
Assuming that $M$ is prime, if $\partial M$ is a torus, then $M$ is necessarily boundary-prime. (There are examples of non-prime manifolds $M$ with toral boundary which are not boundary-prime: Just take the boundary connected sum of a manifold with toral boundary and a manifold, different from the 3-ball, with spherical boundary. But such manifolds are necessarily non-prime.)
Such a manifold still can have compressible boundary, the boundary-compressing $D$ disk in $M$ is necessarily nonseparating. Cutting $M$ open along $D$ results in a manifold $M'$ with spherical boundary. If $M'$ is homeomorphic to the 3-ball, then $M$ itself is a solid torus, $\hat{T}=D^2\times S^1$. Otherwise, attaching $B^3$ along $\partial M'$ results in a closed 3-manifold $N$ which is not $S^3$. Then $M= N\# \hat{T}$. But there is one more possibility, namely, $\partial M$ is incompressible. There are many manifolds like that, for instance, the exterior of any nontrivial knot in $S^3$.
Hence, we obtain the trichotomy:
Suppose that $\partial M$ is homeomorphic to $T^2$. Then one of the following holds:
$M=\hat{T}$.
$M$ is not prime.
$\partial M$ is incompressible.