I know that it holds even if $X$ has infinite dimension, but I am looking for a specific argument in the finitedimensional case.

$\begingroup$ Many such questions boil down to the following trick. Problem: Given an open subset of the $n$skeleton. Can we construct an open subset of the $n+1$skeleton such that the intersection with the $n$skeleton is the given set? So we choose attaching pushouts and we leave out the midpoints of the $n+1$ cells. Then we have a retraction of the $n+1$skeleton without the midpoints to the $n$skeleton. Now we can take the preimage of the given set. $\endgroup$– HenrikRüpingNov 10 '20 at 19:34
The classical (and short) argument is the one in
A. Hatcher: Algebraic topology, Cambridge University Press (ISBN 0521795400/pbk). xii, 544 p. (2002). ZBL1044.55001,
see Proposition A.4 p. 523. I do not think there is any significant simplification taking $X$ finitedimensional.