Deep and interesting theory exists when $F$ is analytic, not just smooth. It was developed by J. F. Ritt in his books on Differential algebra. (In particular, Ritt rigorously defined the things like "singular solutions").
Some parts of the theory also apply to PDE.
Most interesting and useful theory is obtained when $F$ is a polynomial, in which case the set of solution is a "differential-algebraic manifold", and a theory of DA manifolds is parallel to the theory of the usual algebraic manifolds.
For a more modern exposition (including fields of non-zero characteristic) see Kolchin, Differential algebra and algebraic groups.