[edited to insert paragraph on Cornacchia and point-counting]
Continued fractions, or (more-or-less) equivalently the Euclidean algorithm, can be used to find small integer solutions of linear Diophantine equations $ax+by=c$, and integer solutions of quadratic equations such as $x^2-Dy^2=1$$x^2-Dy^2=\pm1$ ("Pell"). Continued fractions in themselves won't find rational points on elliptic curves, but there's a technique using Heegner points that calculates a close real approximation to a rational point, which is then recovered from a continued fraction — this is possible because the recovery problem amounts to finding a small integer solution of a linear Diophantine equation. My paper
Noam D. Elkies: Heegner point computations, Lecture Notes in Computer Science 877 (proceedings of ANTS-1, 5/94; L.M. Adleman and M.-D. Huang, eds.), 122-133.
Noam Dmight have been the first to describe this approach. Elkies: Heegner point computations
Another application of continued fractions is Cornacchia's algorithm to solve $x^2+Dy^2=m$ for large $m>0$ coprime to $D$, given $x/y \bmod m$ which is a square root of $-D \bmod m$. This has an application to Lecture Notes in Computer Sciencecounting points on elliptic curves 877$E\bmod p$ for $E$ such as $y^2 = x^3 + b$ or $y^2 = x^3 + ax$ for which the CM field ${\bf Q}(\sqrt{-D})$ is known: the count (proceedings of ANTS-1, 5/94; L.M. Adlemanincluding the point at infinity) is $p+1-t$ where $t^2+Du^2=4p$ for some integers $t$ and M.-D. Huang$u$, edsand this determines $t$ up to an ambiguity of at most $6$ possibilities that in practice is readily resolved.) The necessary square root mod $p$ is readily found in random polynomial time, 122-133though it is a persistent embarrassment that we cannot extract square roots modulo a large prime in deterministic polynomial time without assuming something like the extended Riemann hypothesis.
might have been Indeed the firstapplication that Schoof gave to describemotivate his polynomial-time algorithm to compute $t$ for any elliptic curve mod $p$ was to recover a square root of $-D \bmod p$ for small $D$ ! (Though this approachwould never be done in practice because the exponent in Schoof's algorithm is much larger than for the randomized algorithm.) The reference for Schoof's paper is
René Schoof: Elliptic Curves over Finite Fields and the Computation of Square Roots $\bmod p$, Math. Comp. 44 (#170, April 1985), 483-494.
A natural generalization of the Euclidean algorithm to higher dimensions is the LLL algorithm and other techniques for lattice basis reduction (LBR), which have found various other Diophantine uses, including some other techniques for finding rational points on elliptic curves; another of my papers describes some of these Diophantine applications of LBR.