Here I will clarify the cohomological issues in Sergei's answer above. For applications to Alexandrov spaces scroll to the end of the post.

I will use Alexander-Spanier cohomology with compact support and $\mathbb Z_2$ coefficients, and the main reference will be Massey's book "Homology and cohomology theory, an approach based on Alexander-Spanier cochains"; I own a Russian translation with insightful comments by Sklyarenko. As usual, using $\mathbb Z_2$ coefficients allows to ignore orientability issues.

Incidentally, as is explained in Massey's book (or in Spanier's "Algebraic topology" text) for locally contractible, locally compact Hausdorff spaces (e.g. for finite-dimensional Alexandrov spaces) the  Alexander-Spanier cohomology coincide with singular and Cech cohomology. 

<b>Lemma.</b> Let $X$ be a locally compact Hausdorff space that 
contains a closed subset $S$ such that $X-S$ is a connected topological $n$-manifold. 
If $U$ is an open subset of $X-S$, then the homomorphism $H_c^n(X, X-U)\to H_c^n(X,S)$ induced by inclusion is an isomorphism.

<i>Proof.</i> By Theorem 1.4 in Chapter 1 of Massey's book, if $A$ is a closed subset of $X$, then there is an isomorphism $H^n_c(X,A)\cong H^n_c(X-A)$.

Theorem 3.21 in Chapter 3 of Massey's book says that if $U$ is an open subset of a manifold $M$,
then the map $H^n_c(U)\to H^n_c(M)$, which associates to a cocycle with compact support in $U$ the same cocycle with support in $M$, is an isomorphism. 

Look at the inclusion $(X, S)\to (X, X-U)$.
Using the above isomorphism, we can identify the induced map $H^n_c(X, X-U)\to H^n_c(X,S)$ with $H^n_c(U)\to H^n_c(X-S)$, which is an isomorphism, as $U$ is open in the manifold $X-S$. QED

Below we denote by $H^n$ the Alexander-Spanier cohomology with arbitrary support;
they coincide with singular cohomology for nice spaces, such as locally contractible, locally compact Hausdorff spaces. Of course, for compact $X$ cohomology with compact support coincide with the the usual cohomology, so we get:

<b>Corollary.</b> If in the assumptions of the Lemma $X$ is compact, then the map
$H^n(X, X-U)\to H^n(X, S)$ induced by inclusion is an isomorphism. QED

Finally, as in Grove-Peterson's paper from Anton's answer, if $X$ is a compact $n$-dimensional Alexandrov space without boundary, and $S$ is the set of non-manifold points, then $S$ has codimension $2$, so long exact sequence of the pair $(X,S)$ shows that $H^n(X,S)\to H^n(X)$ is an isomorphism by inclusion, and we get the isomorphism $H^n(X, X-U)\to H^n(X)$
which implies $H^n(X-U)=0$ by the exact sequence of the pair $(X, X-U)$ because all cohomology in dimension $>n$ vanish. In summary:

<i>
If $X$ is a compact $n$-dimensional Alexandrov space without boundary, and $S$ is the set of non-manifold points, then $H^n(X-U)=0$ for any open subset $U$ of $X-S$.
</i>

Now if $x$ is a point of $X-S$, then $X-x$
deformation retracts to some $X-U$, so $H^n(X-x)=0$ which is what's needed for Sergei's answer.