When is a limit cycle generated by a Hamiltonian oval stable?

Consider a real polynomial $H$ of degree $n+1$ in the plane. A closed, connected component of a level curve $H=t$ is denoted by $\gamma(t)$ and called an oval of $H$. Let $\omega$ be a real 1-form with polynomial coefficients of degree at most $n$. We say that an oval $\gamma(t)$ generates a limit cycle of the perturbed system $dH + \epsilon \omega$ if there exists a family of closed curves $l(\epsilon)$ defined for small $\epsilon$ such that $l(\epsilon)$ is a limit cycle of $dH + \epsilon \omega$ when $\epsilon \neq 0$ and $l(0)=\gamma(t)$. Consider the abelian integral \begin{equation*} I(t) = \int_{\gamma(t)} \omega \end{equation*} The Pontryagin criterion says that if $\gamma(t)$ generates a limit cycle in the above sense then $I(t)=0$. Conversely, if $I(t)=0$ and $I'(t) \neq 0$ then the oval $\gamma(t)$ generates a limit cycle of the perturbed system. What can be said about the STABILITY of limit cycles generated in this way? Can we couple this result to Floquet theory or a related concept in a convenient way?

• Higher Melnikov function described in Perko books gives simillar results for the case of non simple roots of the corresponding abelian integral. I am also interested in these topics please see my question here : In this question the unperturbed system is not Hamiltonian: mathoverflow.net/questions/157966/… – Ali Taghavi Jun 24 '16 at 6:01

Well, it looks like this is simply a matter of a Poincare map. If you look at a proof of Pontryagin's criterion, it turns out that the integral $I(t)$ is the first derivative of the Poincare map with respect to the parameter $\varepsilon$, and then evaluated at $\varepsilon=0$. Indeed, assume you have a cross-section $\Sigma$ of the orbits of $\ker dH$ and parametrize this cross-section by $H|_{\Sigma} : \Sigma \to \mathbb{R}$. For small enough $\varepsilon$ the transverse arc $\Sigma$ is also a cross-section of the orbits of $\ker( dH + \varepsilon \omega)$. Fix a point $t$ on $\Sigma$. Then let $\gamma_{\varepsilon, t}$ be the orbit of $\ker( dH + \varepsilon \omega)$ starting from $t \in \Sigma$ and returning to $\Sigma$ again for the first time at the point $t_1 \in \Sigma$. This is the Poincare first-return map (guaranteed to exist for small enough $\varepsilon$) and we denote it by $P_{\varepsilon}(t):=t_1$, where the map $P_{\varepsilon} : \Sigma' \to \Sigma\,\,\,$ is defined on an open subset $\Sigma' \subseteq \Sigma$. Now, since $\gamma_{\varepsilon, t}$ is a an orbit of $\ker( dH + \varepsilon \omega)$, we can write this fact alternatively as $( dH + \varepsilon \omega)_{{\gamma}_{\varepsilon, t}}(\dot{\gamma}_{\varepsilon, t}) = 0$. Therefore if we integrate the latter equality along $\gamma_{\varepsilon, t}$ we obtain $$0 = \int_{\gamma_{\varepsilon, t}} dH + \varepsilon \omega = \int_{\gamma_{\varepsilon, t}} dH + \varepsilon \int_{\gamma_{\varepsilon, t}} \omega = H|_{t_1} - H|_{t} + \varepsilon \int_{\gamma_{\varepsilon, t}} \omega = t_1 - t + \varepsilon \int_{\gamma_{\varepsilon, t}} \omega.$$ I have slightly abused notation, identifying the endpoints of $\gamma_{\varepsilon, t}$ on the cross-section $\Sigma$ with their coordinates $t$ and $t_1$. Now, by construction $P_{\varepsilon}(t)=t_1$ so the result of the integration above yields $$0 = P_{\varepsilon}(t) - t + \varepsilon \int_{\gamma_{\varepsilon, t}} \omega$$ written also as $$P_{\varepsilon}(t) - t = - \varepsilon \int_{\gamma_{\varepsilon, t}} \omega.$$ Division by $\varepsilon$ takes us to
$$\frac{P_{\varepsilon}(t) - t}{\varepsilon} = - \int_{\gamma_{\varepsilon, t}} \omega.$$ If we let $\varepsilon \to 0$, by continuous dependence on parameters, $\gamma_{\varepsilon, t} \to \gamma(t)$, the latter being the oval orbit of $\ker dH$ through $t$. Therefore the last equality converges to $$\frac{\partial P_{\varepsilon}(t) }{\partial \varepsilon}\Big{|}_{\varepsilon = 0} = - \int_{\gamma(t)} \omega = - I(t).$$ This last conclusion allows us to Taylor expand the Poincare map with respect to $\varepsilon$ near $0$: $$P_{\varepsilon}(t) = t - \varepsilon I(t) + \varepsilon^2 R(t, \varepsilon).$$ Now, a fixed point of the Poincare map, $P_{\varepsilon}(t_{\varepsilon}) = t_{\varepsilon},$ gives rise to a periodic orbit of $\ker( dH + \varepsilon \omega)$ passing through $t_{\varepsilon} \in \Sigma$. So, in search of periodic orbits of $\ker( dH + \varepsilon \omega)$, we try to solve the fixed point equation $$f_{\varepsilon}(t):=\frac{1}{ \varepsilon }(P_{\varepsilon}(t) - t) = - I(t) + \varepsilon R(t, \varepsilon) = 0.$$ The conditions for $I$ you mention are simply the conditions for the implicit function theorem applied to the latter equality, i.e. $f_{\varepsilon}(t) = 0$. Thus, if we find $t_0 \in \Sigma$ such that $I(t_0) = 0$ but $I'(t_0) \neq 0$, then we know that there is a one parameter continuous family of initial conditions $t_{\varepsilon}$ for which $f_{\varepsilon}(t_{\varepsilon}) = 0$, which in its own turn implies the existence of one parameter family of cycles $l(\varepsilon)$ of $\ker( dH + \varepsilon \omega),$ passing through the points $t_{\varepsilon}$ so that $t_{\varepsilon}|_{\varepsilon = 0} = t_0$ and so $l(\varepsilon)$ continuously homotopes to $l(0) = \gamma(t_0)$. Finally, we write the Poincare map as $$P_{\varepsilon}(t) = t - \varepsilon I(t) + \varepsilon^2 R(t, \varepsilon) = t + \varepsilon \big( - I(t) + \varepsilon R(t, \varepsilon)\big) = t + \varepsilon f_{\varepsilon}(t),$$ and its first derivative with respect to $t$ is $$P'_{\varepsilon}(t) = 1 - \varepsilon I'(t) + \varepsilon^2 \frac{\partial R}{\partial t}(t,\varepsilon)=1 + \varepsilon f'_{\varepsilon}(t).$$
Now, the first derivative of $f_{\varepsilon}(t)$ with respect to $t$ evaluated at $t_{\varepsilon}$ is $$g(\varepsilon) = f'_{\varepsilon}(t_{\varepsilon}) = - I'(t_{\varepsilon}) + \varepsilon \frac{\partial R}{\partial t}(t_{\varepsilon},\varepsilon),$$ so we see that $g(\varepsilon)$ is a continuous function with respect to $\varepsilon.$ Assume for instance that $I'(t_0) > 0$. For $\varepsilon = 0$ we obtain $$g(0) = f'_0(t_0) = -I'(t_0) < 0.$$ By continuity, $g(\varepsilon) = f'_{\varepsilon}(t_{\varepsilon}) < 0$ for small enough values of $\varepsilon$. Therefore $$0 < P'_{\varepsilon}(t_{\varepsilon}) =1 + \varepsilon f'_{\varepsilon}(t_{\varepsilon}) < 1,$$ whenever $\varepsilon > 0$ and it is small enough. Analogously, $$P'_{\varepsilon}(t_{\varepsilon}) =1 + \varepsilon f'_{\varepsilon}(t_{\varepsilon}) > 1,$$ for $\varepsilon < 0$ near $0$. The former case implies that the limit cycle is stable, while the latter one implies that the cycle is unstable. The alternative assumption that $I'(t_0) < 0$ is handled in the same way.
Notice that these arguments are more general and it is irrelevant whether $H$ and $\omega$ have polynomial nature.
• Thank you for a very nice answer. If I understand correctly, the punchline is: Assume $\epsilon >0$. Then the limit cycle is stable if $I'(t_0)>0$ and unstable if $I'(t_0)<0$? This seems remarkably simple, but I'm left wondering if there is any notion of semi-stability? Moreover, do you have a good reference to confirm your proof of Pontryagin's criterion? I am a little unsure how much rigour is contained in the one you have provided. (and by $ker(a)=0$ do you just mean $ker(a)$?) – hsp99 Jun 24 '16 at 13:44
• Yes, it is exactly as you say: for small enough $\varepsilon >0$ the limit cycle is stable if $I'(t_0)>0$ and unstable if $I'(t_0)<0$. The function $I(t)$ conveniently encodes a lot of information about the qualitative behavior of the dynamical system in the relevant neighborhood: existence of periodic orbits as well as stability of the latter. Otherwise, to hope for semi-stability, assuming the dynamical system is at least real analytic, or even polynomial, one probably needs to have $I(t_0)=0, \, I'(t_0) = 0$ but $I''(t_0) \neq 0$. BTW, I've already fixed the $\ker$ typo. Thanks. – Futurologist Jun 28 '16 at 0:32
• In terms of literature, I am not too sure where exactly one can see the proof, although I have seen $I(t)$ used in all articles related to the topic of infinitesimal 16th Hilbert problem. But maybe in the books written by some of the experts in the field? – Futurologist Jun 28 '16 at 0:37