Yes. Let $h \in (0,1)$, $Q_h = \frac{1}{W_{\sigma + h} - W_{(\sigma - h) \vee 0}} \int_{(\sigma - h)\vee 0}^{\sigma + h} (H_s - H_{(\sigma - h) \vee 0}) \, dW_s$ and set $M_h = \sqrt{-\log(h)}$. Because $H$ is continuous, and hence, $\lim_{h \to 0} H_{(\sigma - h) \vee 0} = H_{\sigma} $ almost surely, the claim holds if we can prove that $Q_h$ converges to zero in probability.

For arbitrary $\epsilon>0$ \begin{align*}
& P( |Q_h| > \epsilon) = A_h + B_h \quad \text{where} \\
& A_h := P( |Q_h| > \epsilon , |W_{\sigma + h} - W_{(\sigma - h) \vee 0} | > M_h \sqrt{\sigma + h - (\sigma - h) \vee 0 } )\\
& B_h := P( |Q_h| > \epsilon , |W_{\sigma + h} - W_{(\sigma - h) \vee 0} | < M_h \sqrt{\sigma + h - (\sigma - h) \vee 0} )
\end{align*}
By the strong Markov property of BM (given $\mathcal{F}_{(\sigma-h) \vee 0}$), $$
B_h \le P(|W_{\sigma + h} - W_{(\sigma - h) \vee 0} | > M_h \sqrt{\sigma + h - (\sigma - h) \vee 0} ) = \mathcal{N}(0,1)[M_h, \infty] \le e^{-M_h^2 } \;.
$$
By Markov's inequality, the strong Markov property of BM (given $\mathcal{F}_{(\sigma-h) \vee 0}$), and Itô isometry, \begin{align*}
& A_h \le P\left(\left| \frac{1}{\sqrt{\sigma + h - (\sigma - h) \vee 0}} \int_{(\sigma - h)\vee 0}^{\sigma + h} (H_s - H_{(\sigma - h) \vee 0}) \, dW_s \right| > M_h \epsilon \right) \\
& \le E\left(\left| \frac{1}{\sqrt{\sigma + h - (\sigma - h) \vee 0}} \int_{(\sigma - h)\vee 0}^{\sigma + h} (H_s - H_{(\sigma - h) \vee 0}) \, dW_s \right|^2 \right) M_h^{-2} \epsilon^{-2} \\
& \le E\left( \frac{1}{\sigma + h - (\sigma - h) \vee 0} \int_{(\sigma - h)\vee 0}^{\sigma + h} (H_s - H_{(\sigma - h) \vee 0})^2 ds \right) M_h^{-2} \epsilon^{-2}
\end{align*}
By continuity of $H$, we see that $A_h\searrow 0$ and $B_h \searrow 0$, and in turn, $P(|Q_h| > \epsilon) \searrow 0$ as $h \searrow 0$.

Thus, $Q_h$ converges to zero in probability, as required.