I think that the appropriate generalization is that of a tubular neighborhood of a codimension 1 hypersurface in $M$. If the hypersurface $P$ of $M$ is compact, then there exists an open set $U$ in $M$ which contains $P$ and the tangent bundle admits an orthogonal decomposition $$TU=TP \oplus NP$$ where $NP$ is the normal bundle of $P$. An analagous statement to the Gauss lemma shows that this induces a foliation of $U$ whose leaves are hypersurfaces "parallel" to $P$. For a deeper look, I would suggest looking at Lee's new edition of Rirmannian manifolds, specifically the discussion of semigeodesic coordinates.

I think that the appropriate generalization is that of a tubular neighborhood of a codimension 1 hypersurface in $M$. If the hypersurface $P$ of $M$ is compact, then there exists an open set $U$ in $M$ which contains $P$ and the tangent bundle admits an orthogonal decomposition $$TU=TP \oplus NP$$ where $NP$ is the normal bundle of $P$. An analagous statement to the Gauss lemma shows that this induces a foliation of $U$ whose leaves are hypersurfaces "parallel" to $P$. For a deeper look, I would suggest looking at Lee's new edition of Riemannian manifolds, specifically the discussion of semigeodesic coordinates.