Point-set definitions are the most common modern way of defining a geometrical object like a line, but they're not the only way.

In Euclid, lines and circles are primitives.

The axiomatization of Euclidean geometry in Tarski 1999 has only points, betweenness, and congruence as primitives, and sets are not even referred to in the axioms except for the axiom of continuity, which basically says lines are Dedekind-complete. I don't imagine that ultrafinitists would even want this kind of continuity, so they'd probably leave this axiom out. (Even if you interpret the axiom as a sheaf of axioms over first-order formulas defining the relevant sets, that would be an infinite sheaf, which I think would be unacceptable.) Tarski's axioms, interpreted using classical logic, imply that there are infinitely many points, but classical logic isn't the appropriate logic for ultrafinitism. So I don't really see any reason to believe that there's any problem with doing an ultrafinitist geometry this way.

As an example, take the unit circle in the Cartesian plane, defined by a first-order formula (not as a set). I can definitely prove to an ultrafinitist's satisfaction that it contains at least four points (i.e., there are at least four points satisfying that formula). Given a line through the origin O (defined by picking some point P outside the circle that the line goes through), the lack of the axiom of continuity means that it's probably not possible to give an ultrafinitist proof that it intersects the circle (i.e., that there is a point between O and P that satisfies the formula defining the circle). However, one can certainly prove that there exist a point on the line and a point on the circle such that the distance between them is no more than 0.0001. There will be points such that it's neither true nor false that the point is on both the line and the circle.

Even if you're working in a system that has infinities, it's not necessary to describe geometry in terms of point sets. The original presentation of the surreal line didn't use sets, and the surreals are too big to be a ZFC set. In smooth infinitesimal analysis, a curve is not a set of points.

Tarski and Givant, 1999, http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.27.9012

Point-set definitions are the most common modern way of defining a geometrical object like a line, but they're not the only way.

In Euclid, lines and circles are primitives.

The axiomatization of Euclidean geometry in Tarski 1999 has only points, betweenness, and congruence as primitives, and sets are not even referred to in the axioms except for the axiom of continuity, which basically says lines are Dedekind-complete. I don't imagine that ultrafinitists would even want this kind of continuity, so they'd probably leave this axiom out. (Even if you interpret the axiom as a sheaf of axioms over first-order formulas defining the relevant sets, that would be an infinite sheaf, which I think would be unacceptable.) Tarski's axioms, interpreted using classical logic, imply that there are infinitely many points, but classical logic isn't the appropriate logic for ultrafinitism. So I don't really see any reason to believe that there's any problem with doing an ultrafinitist geometry this way.

As an example, take the unit circle in the Cartesian plane, defined by a first-order formula (not as a set). I can definitely prove to an ultrafinitist's satisfaction that it contains at least four points (i.e., there are at least four points satisfying that formula). Given a line through the origin O (defined by picking some point P outside the circle that the line goes through), the lack of the axiom of continuity means that it's probably not possible to give an ultrafinitist proof that it intersects the circle (i.e., that there is a point between O and P that satisfies the formula defining the circle). However, one can certainly prove that there exist a point on the line and a point on the circle such that the distance between them is no more than 0.0001. There will be points such that it's neither true nor false that the point is on both the line and the circle.

Even if you're working in a system that has infinities, it's not necessary to describe geometry in terms of point sets. The original presentation of the surreal line didn't use sets, and the surreals are too big to be a ZFC set. In smooth infinitesimal analysis, a curve is not representable as a set of points.

Tarski and Givant, 1999, http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.27.9012