# Trapped Billiard trajectories on non-convex billiard tables

Let $\Omega$ be a domain in $\mathbb{R}^2$ with smooth boundary. A billiard trajectory is a continuous curve $c: \mathbb{R}\supseteq I \longrightarrow \overline{\Omega}$ such that

1. $c(t) \in \partial \Omega$ only for a discrete set of times $t$
2. if $c(t) \notin \partial \Omega$, then $c|_{[t-\varepsilon, t+\varepsilon]}$ is a straight line for some $\varepsilon>0$.
3. If $c(t) \in \partial \Omega$, then both one-sided derivatives exist and we have $\dot{c}(t+) = \dot{c}(t-) - 2\langle \dot{c}(t-), \mathbf{n}\rangle \mathbf{n}$ where $\mathbf{n}$ is the (exterior or interior) unit normal to $\Omega$.

That is, billiard particles move in straight lines and reflect making the angle of incidence equal to the angle of reflection.

In , the following is proven: If $\Omega$ is convex (and has smooth boundary as always assumed here), then for each $x \in \Omega$ and each velocity vector $v \in \mathbb{R}^2$, there exists a unique billiard trajectory $c$ with $c(0) = x$ and $\dot{c}(0) = v$. Conversely, it is shown that if the boundary is merely $C^2$, then there exists a billiard trajectory that hits the boundary infinitely often in finite time.

It is also standard in the theory of billiards that for almost all $(x, v)$ there exists a unique billiard trajectory starting at $(x, v)$.

Question: What about the smooth, but non-convex case?

It is clear that if a billiard trajectory is "stuck", that is, its boundary-hitting times $t_n$ have a finite limit $t_0$ and hence $c(t_n)$ converges to a point $x \in \partial \Omega$, then the curvature of the boundary must vanish at $x$.

But is there an example for a "stuck" billiard trajectory on a domain with smooth (but non-convex) boundary?

 B. Halpern, Strange Billiard Tables, Transactions of the AMS, 1977

• The paper you cite has the assumption that the curvature doesn't vanish anywhere. If the curvature at a point doesn't vanish, and the third derivative is bounded in a neighborhood of the point, then no trajectory can get stuck at the point. Global failure of convexity isn't important. So, what remains is to consider the cases where the curvature vanishes. Apr 22 '15 at 20:38
• – user25199
May 8 '15 at 8:04
• I know that thread. Douglas Zare claims there that he can construct an example related to the function $e^{-1/x^2}$ but doesn't make it explicit. May 8 '15 at 14:24