The answer to your question is no. Here is a counterexample.
Let $X$ be the CW-complex obtained by attaching a 2-cell to the space $[-1,1]$ via the attaching map $S^1\cong [-1,1]/(-1\sim 1) \longrightarrow [-1,1]$ given by (the continuous extension of) $x\mapsto x\sin(1/x)$.
Then $X\setminus \{0\}$ is homotopy equivalent to an infinite wedge of $S^1$'s.