# Reference request: Invariant sets of dynamical systems

(I should preface this with the disclaimer that this is a slightly elaborated version of a question that I posted onto math se recently to which I received no responses, and have since deleted the question.)

I figured that this question would be among the standard topics covered in texts on dynamical systems, but surprisingly I can find very little on it and am hoping someone can point me in the right direction.

Let $x(t)$ be the trajectory of some dynamical system, say a solution to $\dot{x} = f(x)$.

Lyapunov theory lets one (among other things) reach various conclusions about the behavior of trajectories started near equilibrium points.

Instead, suppose we define a set by $G = \{x | g(x) \geq 0\}$ and we want to know when this is forward invariant. We know that (for instance) $\frac{d}{dt} g(x(t)) \geq 0$ when $g(x(t)) = 0$. I would be interested in any variant of this using instead strict inequalities/ or equalities for any of the conditions.

I know that certain forms of this statement are easy to show, but I can also see that others may be quite subtle. For instance, when relaxing strict inequality constraints, or the situation that $g$ does not have the same left and right hand derivatives. Any reference to a theorem name or book would be helpful.

I am assuming you are interested in multidimensional case $x\in\mathbb R^n$. Let $\Omega$ be the set whose invariance you are interested in estabilishing.
A). One is writing down the conditions for invariance (which essentially boils down to some variant of saying "the vector field points inwards at the boundary of set $\Omega$"
The results that I am aware of for B are for cases where $\Omega$ is ellipsoid, polyhedron, polytope, and cone. You could look for papers by Bitsoris, Blanchini. 'Set invariance in control' in Automatica by Blanchini gives a useful survey (till 1999).