We have been studying a Hamiltonian system that possesses a one-parameter family of periodic orbits, depending on the energy level $h$. We "know" via various non-rigorous means that these periodic orbits are stable for $h<\frac{1}{8}$ and unstable for $h>\frac{1}{8}$ but haven't been able to prove it.
We know that if the linearization possesses a periodic orbit at the critical value $h=\frac{1}{8}$, then this value of $h$ lies on the boundary between stability and instability. We have numerically calculated this periodic orbit using ultra-high-precision arithmetic and a 30th order ODE solver. We have also used Hill's method of harmonic balance to high order in computer algebra to show that $h_{\rm critical}$ agrees with $1/8$ to double precision, using no ODE solves.
If we could simply show that the following ODE has a $2\pi$-periodic orbit, we would have a proof. Does anybody know how find such a solution?
$$\frac{d}{d\theta} \vec{x} = A(\theta) \vec{x},$$Corrected since original posting
$$\frac{d}{dt} \vec{x} = A(t) \vec{x},$$
where $$ A(\theta) = \frac{1}{4\sqrt{17+8 \cos{2 \theta} }} \begin{pmatrix} - \sin{2 \theta} & \frac{7+12 \cos{2 \theta} -4 \cos{4 \theta}-3\sqrt{17+8 \cos{2 \theta} } }{2-2\cos{2 \theta}} \\ \frac{3-4 \cos{2 \theta} -4\cos{4 \theta}-\sqrt{17+8 \cos2 \theta }} {2+2\cos 2\theta}& \sin{2 \theta} \\ \end{pmatrix}. $$$$ A(t) = \left( \begin{array}{cc} -\frac{4 \sin (2 t)}{\sqrt{8 \cos (2 t)+17}} & \frac{8 \cos ^2(2 t)-12 \cos (2 t)+3 \sqrt{8 \cos (2 t)+17}-11}{2 (1-\cos (2 t)) \sqrt{8 \cos (2 t)+17}} \\ \frac{-8 \cos ^2(2 t)-4 \cos (2 t)-\sqrt{8 \cos (2 t)+17}+7}{2 (\cos (2 t)+1) \sqrt{8 \cos (2 t)+17}} & \frac{4 \sin (2 t)}{\sqrt{8 \cos (2 t)+17}} \\ \end{array} \right) $$
This would answer the one big question we left unanswered in this paper also available on my website.