To prove your result we need to rule out the
The first case cannot be circularly stable as soon as $|X|\geq 6$. This can is because if we let $P$ be done by considering circumcircles of triangles with disjoint bases and the common apex . For and $A,B,C,D$ be collinear points in that order, then the second case one can check circumcircles of $PAC$ and $PBD$ intersect in a point not in $X$. We conclude that if $|X|\geq 7$ and $X$ is circularly stable, then either $X$ is collinear or it is dense in the plane.
To classify all circularly stable only when sets with $|X|=6$ that are not collinear we must be in the second case above. Let the points be $A,B,C,D$ as vertices of the parallelogram and $O$ the intersection of the diagonals. Now, the circumcircle of $ABC$ and the line $BD$ must intersect at $D$ so $ABCD$ is inscribed in a circle, i.e. it is a rectangle. So Now looking at the maximum size circumcircle of $ABO$ and the line $BC$ we must have $BC$ tangent to this circumcircle so $ABCD$ must be a finite square. It's easy to check that a square, its center and the point at infinity form a circularly stable setis 6 (The four verties .
There are no circularly stable sets with $|X|=5$ that are not collinear. If there were it would come from the first case. We have the apex $A$ and center three collinear points $B,C,D$. The circumcircle of a rectangle $ABD$ and the line $AC$ intersect in a different pointat infinity), contradiction. On the other hand, as you mention in the OP, any set with $|X|\le 4$ is circularly stable.
