I might have misunderstood the question but, if not, I suggest this.

Denoting, $Z = X \ Y$, take Mellin-Transform expectations of both-sides. Since $E[X^{s-1}] = B(a+s,b)$, this condition you demand amounts to find the solutions of $B(a+s,b)/B(a,b)=1$, which can also be rewritten as:

$\Gamma(a) \Gamma(a+s) = \Gamma(a+b) \Gamma(a+b+s).$

I might have misunderstood the question but, if not, I suggest this.

Denoting, $Z = X \ Y$, take Mellin-Transform expectations of both-sides. Since $E[X^{s-1}] = B(a+s,b)$, this condition you demand amounts to find the solutions of $B(a+s,b)/B(a,b)=1$, which can also be rewritten as:

$\Gamma(a) \Gamma(a+s) = \Gamma(a+b) \Gamma(a+b+s).$

EDIT: I apologize for not having read in details Jon Petersen's 1st answer, which is far more superior to my feeble (though unwanted) rephrasing...