At some point during my research I was confronted with this problem, but I did not dedicate serious time to it. Anyway it stayed in the back of my mind and I'm still interested in hints for it. Application: asymptotic properties of Schroedinger equations, scattering.

You have two convex (compact, smooth, everything) disjoint sets in the plane. Consider a ray starting in the complement of the two sets and bouncing on the boundary of the sets in the usual way, with the ingoing and outgoing rays forming equal angles with the normal to the boundary. Q.: does it always exist a trapped ray which never leaves a ball containing the two sets? This can happen of course if the ray keeps bouncing forever between the two bodies. A trivial example is obtained if the sets have two parallel sides, and the ray is chosen perpendicular to both. Less trivial examples (even strictly convex) can be constructed by choosing a trajectory first, and then joining the dots (i.e., the turning points of the trajectory) with convex curves; with some work and some adjustments in the trajectory, you can produce plenty of examples.

But, is this always the case? given two arbitrary bodies, does it always exist a trapped ray?

EDIT (see Pietro's comment): I mean, another trapped ray besides the 'trivial' trapped ray bouncing between the closest points of the two sets (a general version of the trivial case mentioned above of two parallel sides).

EDIT 2 (quick summary of the discussion for the benefit of future readers): the answer is yes for smooth boundaries and large (in particular, with nonempty interior) sets of initial points. A continuity argument is enough to prove this. If the boundary is non smooth problems may arise. E.g. for two polygons with a couple of facing parallel sides, the only trapped ray is the periodic one.

PS in retrospect, the question was quite elementary, but I really enjoyed to discuss it here :)

  • 5
    $\begingroup$ The starting point given is assigned, right? (if not, a periodic orbit between two points minimizing the distance would do it, as it is notmal to both bodies). $\endgroup$ Sep 10, 2010 at 14:12
  • $\begingroup$ Of course you have always the trivial trapped ray, but I was curious to understand if there are more generic trajectories. As to my 'examples', since a few years elapsed, I would not bet my right hand on their correctness, and if proved wrong I'll edit or remove my question, but the main idea was the following: construct the ray and two boundaries inductively, one piece at the time. Draw the first segment of the ray; at one end, draw a first side of body A; draw the second segment of the ray; at its end, draw a first side of body B; and so on. (continued) $\endgroup$ Sep 10, 2010 at 14:33
  • $\begingroup$ At each step you can adjust the angles and the length of the segment. At the time I was convinced that one could produce bounded trajectories with this method, but I repeat, this was a few years ago and I'm not new to errors :) $\endgroup$ Sep 10, 2010 at 14:35
  • $\begingroup$ I was thinking the same as t3suji at first, but I think I see where I got caught up: Piero wants a trapped ray, not a trapped line! The ray can be untrapped in the other direction. So intuitively a ray can asymptotically approach the two-periodic trajectories and remain trapped. $\endgroup$ Sep 10, 2010 at 14:54
  • $\begingroup$ I believe that for two equal-radii disks, a ray can approach but never reach the shortest segment between them. $\endgroup$ Sep 10, 2010 at 14:57

3 Answers 3


Yes, there is always a trapped ray. The simplest way to see it is to find the path between the two bodies that minimizes length. It is necessarily perpendicular to both surfaces.

EDIT: I see the question was edited to ask for more than this trivial answer, so the new answer: there is a unique trapped ray from any starting point, but it is not trapped in backward time unless it is on the shortest path between the bodies. One can find it by minimizing distance of a zig-zag path alternately touching the two bodies a finite number of times, then passing to a limit.

Here is a generalization: suppose you have a collection of smooth disjoint convex shapes $\{S_i\}$ in the plane arranged in a way that no straight line intersects more than two. Then, for any doubly infinite sequence of indices $ \dots, i_{-1}, i_{0}, i_{1}, \dots $ such that $i_j \ne i_{j+1}$, there is a unique trajectory that intersects the shapes in that order, starting with $S_{i_1}$ in the positive direction and $S_{i_0}$ going backward. If the sequence is periodic, you can find the trajectory just as for the case of two objects. For the infinite case, you can take limits.

Even if the shapes are not convex, as long as they are smooth the trajectories still exist, but they are not necessarily unique. If you want to say something about the case when the obstacles are not smooth, you can extend the rule to make it a non-deterministic dynamical system, where a ray hitting a corner has choices which way to go.

This kind of system is classical dynamical systems, which has been well-understood since early last century. Perhaps someone more knowledgable will supply appropriate references. It is a limiting special case of the theory of the geodesic flow on surfaces of negative curvature.

In response to a comment, here is some more detail (that doesn't itself fit into a comment). The question was about stability and how to prove convergence under the limiting process.

To prove existence, you don't need stability: just take a sequence of longer and longer rays, and choose a convergent subsequence. This exists because of compactness of the set of possible initial directions.

To prove uniqueness: this follows from the hyperbolicity of the flow. Think of the convex obstacles as trick mirrors that make you look skinny, cylinders with a convex cross-section. The convexity implies that reflected rays diverge at least as fast as they would from a flat mirror. Successive reflected images of the two mirrors in each other get thinner and thinner, so they narrow down to a unique point. (In the three-dimensional picture, they're also narrowing vertically, just at the relatively slow rate at which images shrink with distance in Euclidean space rather than at the exponential rate resulting from mirrors that are convex to 2nd order).

One way to formalize the discussion above is by use of triangle comparison theorems. Double the complement of the convex bodies to make a surface. The surface can be smoothly approximated by a surface of nonpositive curvature if it's comforting, but that's not technically necessary; the (intuitively obvious) statement about image sizes above become cases of the Toponogov comparison theorem.

  • 1
    $\begingroup$ @Bill: I think Piero is looking for "non-trivial" solutions =) (See his comment, 3rd on the main question). If I interpret it right, he is asking whether one can always find a ray that asymptotically approach the trivial solution. $\endgroup$ Sep 10, 2010 at 14:57
  • 1
    $\begingroup$ Yes, I edited my response after seeing his comment and reading more carefully. The unique infinite solution from any given point is asymptotic to the trivial solution. I know that Morse and Hedlund addressed this kind of thing and introduced symbolic dynamics to describe the trajectories, but I don't know the references firsthand and I'm hazy as to the histoy. It's been folded into the theory of Anosov flows. $\endgroup$ Sep 10, 2010 at 15:08
  • $\begingroup$ Bill, the method sounds convincing, but how do you prove convergence when the number of hits goes to infinity? seems pretty unstable, can you please elaborate a little? $\endgroup$ Sep 10, 2010 at 16:47
  • 1
    $\begingroup$ One could think of this as a sort of intermediate-value-theorem proof. Some directions end up on the same side as you started, and others go between the two bodies. Between the two you get trapped rays. (It's fairly easy to show that the two sets of directions are open intervals.) It's not quite true that you can start at any starting point: you need a reasonable view between the two bodies to ensure that both sets of directions are non-empty. But any "reasonable" starting point will be OK. $\endgroup$
    – gowers
    Sep 10, 2010 at 17:08
  • $\begingroup$ But, suppose body A has a part of the boundary which is a straight segment I, and body B is just the reflection of A across a line parallel to I. Now if the starting point is in the rectangle between I and its mirror, you have a periodic orthogonal trajectory if the initial direction is orthogonal to I, and in all other cases the trajectory is unbounded. For initial points outside the rectangle, the ray can not bounded. Am I wrong? $\endgroup$ Sep 10, 2010 at 17:30

This is not an answer to your specific question, but it reminded me of Janos Pach's related and as-yet unsolved enchanted forest problem: If you light a match in a forest of disjoint circular mirror trees, will light escape to infinity? It is cited, for example, on p.19 in H. T. Croft, K. J. Falconer, and R. K. Guy, Unsolved Problems in Geometry, Springer, New York, 1991. Here is an image I made with a student while investigating this question.
          Mirror Trees

  • 13
    $\begingroup$ +1 for the beautiful picture $\endgroup$ Sep 10, 2010 at 14:41
  • 3
    $\begingroup$ :-) $\mbox{}\;$ $\endgroup$ Sep 10, 2010 at 14:53
  • 4
    $\begingroup$ This is an inhomogeneous Lorentz gas. $\endgroup$ Jan 9, 2012 at 13:00

A reasonable conjecture (assuming smoothness and strict convexity of the two bodies) seems to me that any bounded bouncing ray necessarily lies completely on one side of the minimizing connecting line, and is asypmtotic to it as $t\to\infty.$ That there is at least one such ray on each side seems also true. Fix a parallel line $r$ to the minimizing line, still connecting the two bodies. Perturb the periodic trajectory of an angle $\epsilon$ from one of its two bouncing points, from the side where $r$ is. In a finite number of bouncings you will hit the line $r$ with a based vector $v_\epsilon$. Take a limit $ v_0$ of a subsequence of the $v_\epsilon$ as $\epsilon\to0$: I'd say that starting with initial point and velocity $-v_0$ you will get a ray trapped between the two lines and the bodies (this seems easy to show) and having the minimizing segment as $\omega$-limit.

(As to the tagging issue, checking the tag list I think the most suitable are: dynamical-systems and convex-geometry. Actually, even more precise should be billiard (if not pinball); maybe we could create it).

EDIT: actually, there is a whole open set of initial points that admit a ray asymptotic to the minimizing segment (let's call $a$ and $b$ its endpoints, belonging resp. to the convex $A$ and $B$).

Precisely, for $x\notin A\cup B$ let $x'\in A$ be the point of minimal distance to $x$. Assume that $x$ sees the whole arc $\Gamma$ of $\partial A$ connecting $a$ and $x'$ (here "sees $\Gamma$ " of course means that the convex hull of $x$ and $\Gamma$ does not meet $B$). It is easy to show that there exists a point $y$ on the arc $\alpha$ such that a ray from $x$ to $y$ generates a ray asymptotic to the segment $[a,b]$. (I think it is also unique, and that this construction produces all the bounded rays.)

  • $\begingroup$ While sitting on the dentist' chair it occurred to me that if the convex bodies have two parallel facing sides, any trajectory starting in a non orthogonal direction escapes after a finite number of bounces. So probably some strict convexity assumption is necessary, isn't it? $\endgroup$ Sep 10, 2010 at 16:45
  • 2
    $\begingroup$ Only the assumption of smoothness is necessary. Any trajectory that actually hits one of the parallel facing flats escapes after finitely many bounces. The trajectories that are trapped skirt the edge, coming closer and closer without actually arriving at the facing flats. $\endgroup$ Sep 10, 2010 at 17:40
  • 1
    $\begingroup$ Ok the problem seems essentially solved. Sorry Pietro, the system requires to accept at most one answer; I feel Bill's is more detailed $\endgroup$ Sep 10, 2010 at 17:48
  • $\begingroup$ Actually I find Bill's variational argument more elegant, and it's certainly more general (it also holds for the more dimensional analog). The shooting method in D2 still has some advantages: I see now there is an easy strict monotonicity argument showing that for each point that sees both bodies there are exactly two bounded rays. They start with the limit directions of the rays that eventually pass between the two bodies, and they are asymptotic to the minimizing connecting segment. So this also give a complete characterization of all bounded rays. $\endgroup$ Sep 11, 2010 at 13:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.