Here's an idea which might work (I'm not sure about the technical details).
Assume $M$ is closed. Choose a self-indexing Morse function $f\: M \to \Bbb R$. This will give a handlebody structure on $M$. If the support of $c$ is contained inside the $p$-skeleton of this handlebody, then we take $U$ to be the $p$-skeleton. This will do the job in this case.
If not, we may still consider the problem of intersecting the (image of each of) the simplices of $c$ with the co-cores to the handles of index $> p$. If this intersection is transversal, then $c$ will miss these co-cores. We can then cut out small tubular neighborhoods of the co-cores, and what's left will be an open set $U$ which satisfies your condition.
In the general case, we can ask whether it's possible to make a slight perturbation the Morse function $f$ to a new self-indexing Morse function $g\: M \to \Bbb R$ in such a way that $c$ misses the co-cores of the handles of the new handlebody of index $> p$. It seems to me that some version of transversality will enable one to arrange this condition. However, this is where I don't see the details.