# Sets invariant under sections

Let $X$ be a compact in the Polish space (metric, complete, separable) and $G\subseteq X\times X$ is open. For $x\in X$ we define the section of $G$: $$s(x) = (y\in X|\langle x,y \rangle \in \bar{G}).$$

Here $\bar{G}$ is a closure of $G$.

The set $A'\subseteq X$ is invariant if for all $x\in A'$ holds $s(x)\subset A'$. How to verify if there are non-empty invariant subsets of a given compact $A\subset X$? Maybe there are known equivalent problems?

It will be even helpful in the case $X = [0,1]$.

I also asked it here, however haven't received an answer.

-
I'm not sure what $\bar{G}$ means (I'd guess closure, but could be complement), but it seems like what you want to look at is the transitive closure $\leq$ of $\bar{G}$. Then there should be a nontrivial invariant set iff there are $x$ and $y$ such that $x \not\leq y$. Maybe you want to put other restrictions on the sorts of sets you want to consider? – Clinton Conley Jun 8 '11 at 16:51
@Clinton Conley: I've edited - I am looking for invariant subsets of given compacts. And $\bar{G}$ means the closure. – Ilya Jun 8 '11 at 19:25
Gortaur: Considering the non-perfect case, if $A$ is a set of finitely many isolated points, and $G=A\times A$ then $G=\bar G$ and the condition holds. You might want to specify perfect polish spaces, not just any polish space. – Asaf Karagila Jun 8 '11 at 20:16
@Asaf Karagila: $G$ and $A$ are given and we can do nothing with the choice of them. – Ilya Jun 9 '11 at 5:18
@Gortaur: I believe that you need to give some limitations on the space (namely, perfect, to exclude the end cases of isolated points) as well consider some "trivial" cases and try to see if some information can be said on $G$ (for example, you might want to have an example that $X\subseteq Dom(\bar G)\cup Rng(\bar G)$, or something else added). After you have investigated some basic (or even common) cases you can try to see how the general case holds. – Asaf Karagila Jun 10 '11 at 16:27

Your setup defines set-valued dynamics on $X$. More precisely, you have $$X \stackrel{p}{\leftarrow} \overline{G} \stackrel{q}{\rightarrow} X$$ where $p$ and $q$ are the obvious projection maps. The set-valued transformation in question sends $x \in X$ to the subset $qp^{-1}(x) \subset X$. I'll write $F = qp^{-1}$ for convenience, noting that $F:X \to 2^X$

Now, the set $I \subset X$ is invariant in your sense precisely when $F(I) \subset I$. There is a ton of literature on set-valued dynamics, and I'm certainly in no position to do the field justice. My favorite reference is Ethan Akin's work: he has an entire book called "the general topology of dynamical systems" which would be relevant. On his webpage you can find a ton of references: I'd start with the "Tourist's Guide" survey paper, it is extremely well-written.

If you don't care much about the precise structure invariant sets sitting inside some compact subset $A \subset X$, you can use the Conley index theory to get sufficient conditions for the existence of a (nonempty) maximal invariant set $I \subset A$ under the (minor) additional assumption that $A$ isolates $I$. Briefly, define the exit set $E \subset A$ to be the collection of those $a \in A$ whose forward orbit sets $F^n(a)$ eventually leave $A$. Now, if the homotopy type of the quotient $A/E$ (called the homotopy conley index) is non-trivial (again, we need some niceness properties on $A$ and $E$, maybe an ANR pair or something) then you are guaranteed a non-empty invariant set $I \subset A$. Sadly, this is not if and only if: it is possible to have invariant sets inside $A$ with trivial Conley index.

-
I guess, your map $F$ coincides with a $s$ in OP treated as a map. I have read several papers on a set-valued dynamics, but unfortunately have never found conditions on the existence of such invariant sets. You references - and especially on the Conley index theory - seems very nice, so thanks a lot. – Ilya Jul 17 '13 at 8:18

Please forgive me if this is wrong but it seems to me that as stated there are no non-empty invariant sets since $G = \bigcup$ ($B_{\alpha}$ $\times$ $B_{\beta}$) where $B_{\alpha}$ and $B_{\beta}$ are open balls in $X$. Then $\forall x \in A$ $s(x) =$ $\overline{\bigcup B_{\gamma}}$ or $\varnothing$.

-
This seems to say that $G$ is a rectangle, but there's nothing in the problem to imply that. – Andreas Blass Jun 8 '11 at 17:54
Sorry for the typo, but I think judging by Clinton Conley's comment the answer is academic anyway... – George Lazou Jun 8 '11 at 18:00
In fact it is more than a typo... please forgive my nonsense... if only you could delete :( – George Lazou Jun 8 '11 at 18:45
@Condor: thank you, but it's easy to provide a counterexample. – Ilya Jun 8 '11 at 19:26