Here's an approach which might work (I'm not sure about the correctness of this.)
1) Assume $M$ is closed. Choose a triangulation $T$ of $M$.
If the support of $c$ is contained inside the $p$-skeleton of this triangulation, then
we take $U$ to be a small regular neighborhood of $p$-skeleton. This will do the job in this case.
2) If not, let's consider the dual triangulation $T^\ast$. We can ask whether or not
$c$ meets the $(n-p-1)$-skeleton of $T^\ast$. If it doesn't we can let $U$ be the effect
of deleting the $(n-p-1)$-skeleton of $T^\ast$ from $M$. Then it seems to me that
$U$ has the correct property in this case.
3 ) More generally, we can ask whether there exists a triangulation $T$
satisfying the (2). It seems to me that it should be possible to slightly modify $T$, say by general position, so that (2) holds.
Remark: I originally conceived of a version of this using Morse theory, but then realized that I had to retract it because I got confused. Perhaps it's possible to
find a Morse function $f\: M \to \Bbb R$ such that $c$ misses the $(n-p-1)$-skeleton
of the handlebody defined by $-f$? If so, we can define $U$ to be $M$ with this skeleton removed.