Let $S$ be any regular CW decomposition of (a space homeomorphic to) the $n$-sphere, and consider a cell $\sigma$ of dimension $d \in \{0,\ldots,n\}$. Let $S'$ be the regular CW complex which remains after the open star of $\sigma$ is removed (that is, we remove the interior of $\sigma$ along with interiors of all cells $\tau$, if any, which contain the closure of $\sigma$ in their boundary).

I'd like to be able to make the following claim as a small part of a proof I'm writing:

$S'$ is contractible.

Is this true? It certainly seems reasonable, and I could see a straightforward proof in the case where $S$ is a finite triangulation. If $S$ has only finitely many cells to begin with, I could obtain a triangulation by barycentric subdivision and then the desired statement holds. I'd really like to avoid that, but I worry about pathological examples like the Alexander horned sphere, etc.

So if I can't make the claim above, are there reasonable hypotheses to impose on $S$ (like: only finitely many cells allowed) which will make my desired claim true?

Edit of course there are only finitely many cells, see Mincong Zeng's comment. But do we need anything stronger in case the claim doesn't hold as-is?