First of all, $\dim_{H} (A) = \alpha$ iff $ H^k(A)=\infty$ for all $k<\beta$ and $H^k(A) = 0$ for all $k>\beta$. Then $H^\alpha(A) = \infty$ for all $\alpha \in (0,\beta)$.
If $A$ is closed then by Theorem 5.4 from [1] there is a compact $K\subset A$ such that $0<H^\alpha(K)<\infty$.
More generally, if $A$ is Souslin then by Theorem 5.6 from [1] again there is a compact $K\subset A$ such that $0<H^\alpha(K)<\infty$ (as @SeverinSchraven noted).
If $A$ is not Souslin then (at least assuming AC and CH) it may happen that $H^\beta(B) = 0$ for any $B\subset A$ and any $\beta \in (0, \alpha)$. In order to justify this claim let me add adopt the argument from a comment by fedja to a closely related post, where an interesting paper [2] is also mentioned.
Let $C$ denote the standard Cantor set in $[0,1]$ and take $\alpha = \dim_H(C) = \frac{\ln 2}{\ln 3}$. Take $\beta \in (0,\alpha)$.
Consider the family $\mathcal E$ of all $G_\delta$-subsets $E \subset C$ such that $H^\alpha(E)=0$. Then for any countable family $(E_i)_i\in \mathcal E$ the set $C \setminus \bigcup_{i=1}^\infty E_i$ has infinite $\beta$-dimensional Hausdorff measure and in particular is not empty.
Assuming AC+CH one can enumerate the family $\mathcal E$ using ordinals as $\mathcal E = \{E_\gamma : \gamma < \omega_1\}$, where $\omega_1$ is the first uncountable ordinal.
For each $\gamma < \omega_1$ take $x_\gamma \in C \setminus \bigcup_{\lambda < \gamma} E_\lambda$. Now we finally define the set $A:= \{x_\gamma : \gamma < \omega_1\}$.
If $H^\alpha(A)$ was zero then one could cover $A$ with $E_\gamma$ for some $\gamma$ (see e.g. Theorem 1.6 in [1]). This contradicts the definition of $A$, hence $H^\alpha(A)>0$.
Now if $B \subset A$ and $H^\beta(B)\in (0,\infty)$ then $H^\alpha(B)=0$ and then there exists $\gamma<\omega_1$ such that $B \subset E_\gamma$, hence
$$
H^\beta(B) = H^\beta(B\cap E_\gamma) \le H^\beta(A\cap E_\gamma) = 0
$$
since the intersection $A\cap E_\gamma$ is at most countable for any $\gamma < \omega_1$.
References.
The Geometry of Fractal Sets by K.J. Falconer (1985)
Finding subsets of positive measure by Bjørn Kjos-Hanssen and Jan Reimann (2014) https://arxiv.org/abs/1408.1999