Your setup defines *set-valued dynamics* on $X$. More precisely, you have
$$X \stackrel{p}{\leftarrow} \overline{G} \stackrel{q}{\rightarrow} X$$
where $p$ and $q$ are the obvious projection maps. The set-valued transformation in question sends $x \in X$ to the subset $qp^{-1}(x) \subset X$. I'll write $F = qp^{-1}$ for convenience, noting that $F:X \to 2^X$

Now, the set $I \subset X$ is invariant in your sense precisely when $F(I) \subset I$. There is a ton of literature on set-valued dynamics, and I'm certainly in no position to do the field justice. My favorite reference is Ethan Akin's work: he has an entire book called "the general topology of dynamical systems" which would be relevant. On his webpage you can find a ton of references: I'd start with the "Tourist's Guide" survey paper, it is extremely well-written.

If you don't care much about the precise structure invariant sets sitting inside some compact subset $A \subset X$, you can use the Conley index theory to get sufficient conditions for the existence of a (nonempty) maximal invariant set $I \subset A$ under the (minor) additional assumption that $A$ isolates $I$. Briefly, define the *exit set* $E \subset A$ to be the collection of those $a \in A$ whose forward orbit sets $F^n(a)$ eventually leave $A$. Now, if the homotopy type of the quotient $A/E$ (called the *homotopy conley index*) is non-trivial (again, we need some niceness properties on $A$ and $E$, maybe an ANR pair or something) then you are guaranteed a non-empty invariant set $I \subset A$. Sadly, this is not if and only if: it is possible to have invariant sets inside $A$ with trivial Conley index.

perfectpolish spaces, not just any polish space. – Asaf Karagila Jun 8 '11 at 20:16somebasic (or even common) cases you can try to see how the general case holds. – Asaf Karagila Jun 10 '11 at 16:27