For a specific problem, there is a finite check to decide whether any solutions exist to $x^2 - a y^2 = b.$ Find the minimum positive integrs $u,v > 0$ such that $u^2 - a v^2 = 1.$ Any solution to your problem, say with $x,y$ positive, creates an infinite sequence of solutions by $$ (x,y) \mapsto (ux + av y, \; vx + u y). $$ This will increase the entries. In the other direction, that of decreasing one or both entries, $$ (x,y) \mapsto (ux - av y, \; -vx + u y). $$ Repeating this mapping gets us to a solution with both $x,y> 0$ but one of the entries in the left neighbor nonpositive, either $$ ux - ay \leq 0 \; \; \; \mbox{OR} \; \; \; -vx + uy \leq 0. $$ If you draw some picture, including the hyperbola $x^2 - a y^2 = b,$ you see how one or the other of $ux \leq ay$ or $uy \leq vx$ gives a bounded arc of the hyperbola. If there are no integer solutions in that arc, there are no solutions at all.
More generally, one may draw the Conway topograph for an indefinite form $a x^2 + bxy + c y^2.$ His "climbing lemma" shows how we need investigate only a finite region of the diagrma to decide whether there is a (primitive) solution to $a x^2 + bxy + c y^2= n.$