Here is a variant of an example due to Lurie (as far as I can tell) [HTT, Counterexample 6.5.4.2] showing that the proper base change theorem [Spaltenstein, Proposition 6.20] does not hold for unbounded complexes, even on compact Hausdorff spaces. In particular, as condition 6.14(2) can be replaced by 6.14(a)–(f) or condition (*), the example shows that a class $\mathfrak S$ with these conditions does not always exist.
In fact, Lurie uses this as one of the motivations why the natural notion of $\infty$-topos is defined in terms of sheaves instead of hypersheaves; see [HTT, Warning 6.5.4.1]. I'm not entirely sure what this translates to in the language of derived categories, but I suspect the "correct" category will somehow look more like "sheaves of spectra" than like chain complexes in $\operatorname{Sh}(X,\mathbf Z)$.
For self-containedness, let me translate the example to the language of Spaltenstein. It turns out that this is a little more involved, roughly because $\operatorname{Hom}(\mathbf Z^{(S)},\mathbf Z^{(T)})$ is harder to compute than $\operatorname{Map}(S,T)$ for sets $S$ and $T$; see the remark after the example.
Example. Let $Q = [0,1]^{\mathbf N}$ be the Hilbert cube (where $\mathbf N = \{0,1,\ldots\}$). For $i \in \mathbf N$, write $Q_i = [0,1]^{\mathbf N_{> i}}$, so that $Q = [0,1]^{\{0,\ldots,i\}} \times Q_i = [0,1]^{i+1} \times Q_i$. For $i \in \mathbf N$, define opens in $X = Q \times [0,1]$ by
\begin{align*}
U_i &= (0,1)^i \times [0,1)\times Q_i \times [0, 2^{-i}),\\
V_i &= (0,1)^i \times (0,1] \times Q_i \times [0, 2^{-i}),\\
\end{align*}
and write $U_{i,0} = U_i \cap (Q \times \{0\})$ and $V_{i,0} = V_i \cap (Q \times \{0\})$. Then we have $U_i \cup V_i \subseteq U_{i-1} \cap V_{i-1}$ for all $i \geq 1$, and this induces an equality
$$U_{i,0} \cup V_{i,0} = U_{i-1,0} \cap V_{i-1,0}.\tag{1}\label{1}$$
Consider the complex $C^\bullet \in \operatorname{Ch}^{\leq 0}(X,\mathbf Z)$ given by
$$\cdots \to \mathbf Z_{U_i} \oplus \mathbf Z_{V_i} \to \cdots \to \mathbf Z_{U_1} \oplus \mathbf Z_{V_1} \to \mathbf Z_{U_0} \oplus \mathbf Z_{V_0},$$
where the differentials are all given by the matrix $\big(\begin{smallmatrix}1 & 1 \\ -1 & -1\end{smallmatrix}\big)$. Consider the base change square
$$\begin{array}{ccc}Q \times \{0\} & \stackrel{q'}\to & X \\ \!\!\!\!\!{\scriptsize f'}\!\downarrow & & \downarrow\!{\scriptsize f}\!\!\! \\ \{0\} & \stackrel q\to & [0,1].\!\end{array}$$
We claim that the natural map $q^*Rf_! C^\bullet \to Rf'_!q'^* C^\bullet$ is not an isomorphism in $D(\{0\},\mathbf Z) = D(\mathbf Z)$. In fact, we will show that the left hand side is $\mathbf Z[-1]$ and the right hand side $\mathbf Z[0]$, so the map is zero as $\operatorname{Hom}_{D(\mathbf Z)}(\mathbf Z[-1],\mathbf Z) = \operatorname{Ext}^1(\mathbf Z,\mathbf Z) = 0$.
For the right hand side, restricting to $Q \times \{0\}$ gives the complex
$$\cdots \to \mathbf Z_{U_{i,0}} \oplus \mathbf Z_{V_{i,0}} \to \cdots \to \mathbf Z_{U_{1,0}} \oplus \mathbf Z_{V_{1,0}} \to \mathbf Z_{U_{0,0}} \oplus \mathbf Z_{V_{0,0}},$$
which is acyclic in negative degree by \eqref{1} and the Mayer–Vietoris sequences
$$0 \to \mathbf Z_{U \cap V} \stackrel{\big(\begin{smallmatrix}1 \\ -1\end{smallmatrix}\big)}\longrightarrow \mathbf Z_U \oplus \mathbf Z_V \stackrel{(1\ 1)}\longrightarrow \mathbf Z_{U \cup V} \to 0.$$
Since $U_{0,0} \cup V_{0,0} = Q \times \{0\}$, the Mayer–Vietoris sequence then shows that the natural quotient
$$q'^*C^\bullet \to \mathbf Z_{Q \times \{0\}}[0]$$
is a quasi-isomorphism. Thus, $Rf'_!q'^* C^\bullet = Rf'_!\mathbf Z[0] = \mathbf Z[0]$ since $Q$ is compact and contractible.
For the left hand side, we have to do a little work to show that $C^\bullet$ is Postnikov complete, which is the only way I know how to compute cohomology of complexes that are unbounded below on an infinite-dimensional space. (Please let me know if you know a different method!)
For each finite set $I \subseteq \mathbf N \cup \{\infty\}$, write $\pi_I \colon X \to [0,1]^I$ for the projection (where $\infty$ is the separate factor $[0,1]$ in $X = Q \times [0,1]$). Consider the set $\mathfrak P$ of sets of the form $\pi_I^{-1}(A)$, where $I \subseteq \mathbf N$ is finite and $A = \prod_{i \in I} A_i$ is a product of intervals (open, half-open, or closed). Note that $U_i$, $V_i$, $U_i \cup V_i$, and $U_i \cap V_i$ are in $\mathfrak P$ for all $i$, as are the sets
$$W_i = \big(U_i \cap V_i\big)\setminus\big(U_{i+1} \cup V_{i+1}\big) = (0,1)^{i+1} \times Q_i \times [2^{-i-1}, 2^{-i}).$$
Since the intersection of two intervals is an interval, the set $\mathfrak P$ is closed under finite intersections. For $x \in X$, write $\mathfrak P_x = \{A \in \mathfrak P\ |\ x \in A^\circ\}$, and note that this forms a neighbourhood basis of $x$ (it will be convenient not to restrict solely to open neighbourhoods). Write $\mathfrak B = \operatorname{Open}(X) \cap \mathfrak P$ and $\mathfrak B_x = \mathfrak B \cap \mathfrak P_x$; these form a basis for the topology and for the open neighbourhoods of $x$ respectively. Finally, note that if $A \in \mathfrak P$, then $\bar A \in \mathfrak P$.
Lemma. If $A \in \mathfrak P$ and $x \in X$, then $H^i(V,\mathbf Z_A) = 0$ for all $i > 0$ and $V \in \mathfrak P_x$ sufficiently small.
In particular, the opens $\mathfrak B$ and the sheaves $\mathbf Z_A$ for $A \in \mathfrak P$ satisfy condition 3.12(1) in [Spaltenstein].
Proof. Let $B = \bar A$ and $D = B \setminus A$. If $A = \pi_I^{-1}\big(\prod_{i\in I} A_i\big)$ with $\bar A_i = [a_i,b_i]$, then
$$D \subseteq \partial A = \pi_I^{-1}\big(\prod_{i \in I} \{a_i,b_i\}\big).$$
Call the faces of $D$ the subsets of the form $\pi_i^{-1}(\{a_i\})$ or $\pi_i^{-1}(\{b_i\})$. Consider those $V \in \mathfrak P_x$ such that $V \cap \partial A$ only meets the faces containing $x$; concretely this can be accomplished by taking $V$ contained in $\pi_I^{-1}\big(\prod_{i \in I}[x_i-\varepsilon,x_i+\varepsilon]\big)$ for $\varepsilon$ such that $\varepsilon < \lvert x_i-a_i\rvert$ whenever $x_i \neq a_i$, and likewise for $b_i$. This means that
$$V \cap D = V \cap \bigcup_{i \in F} \pi_i^{-1}(\{x_i\}) \qquad \text{where } F = \{i \in I\ |\ x_i\in\{a_i,b_i\}\}.$$
Thus, $V \cap D$ is either empty (if $F = \varnothing$) or contractible. Since the locally closed immersion $A\cap V \to V$ factors via the closed immersion $B \cap V \to V$, we get
$$R\Gamma(V,\mathbf Z_A) = R\Gamma(V,\mathbf Z_{A \cap V}) = R\Gamma(B \cap V,\mathbf Z_{A \cap V}).$$
If $F = \varnothing$, then $A \cap V = B \cap V$, so we conclude that $R\Gamma(V,\mathbf Z_A) = \mathbf Z[0]$ since $B \cap V$ is contractible. Likewise, if $F \neq \varnothing$, then $V \cap D$ is contractible, so the map $R\Gamma(B \cap V,\mathbf Z_{B \cap V}) \to R\Gamma(B \cap V,\mathbf Z_{D \cap V})$ is an isomorphism, so we conclude $R\Gamma(B \cap V,\mathbf Z_{A \cap V}) = 0$. $\square$
Going back to the complex $C^\bullet$ above, we have $\mathscr H^{-i}(C^\bullet) \cong \mathbf Z_{W_i}$ by Mayer–Vietoris, since the kernel of $d^{-i}$ is $\mathbf Z_{U_i \cap V_i}$ and the image of $d^{-i-1}$ is $\mathbf Z_{U_{i+1}\cup V_{i+1}}$. Thus the lemma above and [Spaltenstein, Proposition 3.13] (see also [Stacks, Tag 0D63]) imply that
$$C^\bullet \to \underset{\substack{\longleftarrow \\ n}}{\operatorname{holim}} \tau_{\geq -n}C^\bullet$$
is an isomorphism in $D(X,\mathbf Z)$. Then [Stacks, Tag 0D60] shows that
$$R\Gamma\big(Q \times [0,a),C^\bullet\big) = \underset{\substack{\longleftarrow \\ n}}{\operatorname{holim}} R\Gamma\big(Q \times [0,a), \tau_{\geq -n} C^\bullet\big),$$
so we have to compute $R\Gamma(Q \times [0,a),\tau_{\geq -n}C^\bullet)$. For $n < 0$ we have $\tau_{\geq -n} C^\bullet = 0$. By [Stacks, Tag 08J5] we have distinguished triangles
$$\mathscr H^{-i}(C^\bullet)[i] \to \tau_{\geq -i} C^\bullet \to \tau_{\geq -i+1} C^\bullet.\tag{2}\label{2}$$
Recall that $\mathscr H^{-i}(C^\bullet) = \mathbf Z_{W_i}$. Thus the same computation as in the lemma (plus the proper base change theorem and the computation of $R\Gamma_c\big((0,1)^{i+1},\mathbf Z\big)$) gives
$$R\Gamma\big(Q \times [0,a),\mathbf Z_{W_i}\big) = \begin{cases}0, & a > 2^{-i},\\ \mathbf Z[-i-1], & 2^{-i-1} < a \leq 2^{-i}, \\ 0, & a \leq 2^{-i-1}.\end{cases}$$
Indeed, in the first case, the closed interval $[2^{-i-1},2^{-i}]$ is contained in $[0,a)$, so again the inclusion
$$(0,1)^{i+1} \times Q_i \times \{2^{-i}\} \to (0,1)^{i+1} \times Q_i \times [2^{-i+1},2^{-i}]$$
induces an isomorphism on compactly supported cohomology. In the second case, $W_i \cap (Q \times [0,a))$ is closed and contractible in $Q \times [0,a)$, and in the third case it is empty.
Given $a \in (0,1]$, there is a unique $i \in \mathbf N$ with $2^{-i-1} < a \leq 2^{-i}$. From \eqref{2} we thus conclude that
$$R\Gamma\big(Q \times [0,a),\tau_{\geq -n} C^\bullet\big) = \mathbf Z[-1]$$
as soon as $n \geq i$. Thus, the cohomology of $\tau_{\geq -n}C^\bullet$ is eventually constant for all $n$ and $a$, and we conclude $R\Gamma(Q \times [0,a),C^\bullet) = \mathbf Z[-1]$. Since $R^if_!C^\bullet = R^if_*C^\bullet$ is the sheafification of $U \mapsto H^i(f^{-1}(U),C^\bullet)$ [Stacks, Tag 0D5X] and sheafification preserves stalks, we conclude that
$$q^*Rf_!C^\bullet = \mathbf Z[-1],$$
which has cohomology only in degree $1$, and does not agree with $Rf'_!q'^*C^\bullet$. $\square$
Remark. The example of [HTT, Counterexample 6.5.4.2] is much easier, because it works with sheaves of spaces instead of chain complexes of sheaves of abelian groups. The point is that the sheaf of sets $\pi_0(\mathscr F)$ of Lurie's example has empty global sections, so $\mathscr F$ itself cannot have global sections. But in abelian categories, nonzero objects can of course map to the zero object, and the vanishing of $q^*Rf_*C^\bullet$ was substantially more work. There might be shortcuts to the above argument using simplicial or homotopical techniques.
Remark. One can immediately see that the Hilbert cube is not locally finite-dimensional: opens around an 'interior' point (no coordinate is $0$ or $1$) have higher and higher cohomological dimension, already for compactly supported cohomology with constant coefficients.
On the other hand, checking (*) directly in this case is a little unclear, but the failure of proper base change shows that it cannot be true. So Spaltenstein wasn't wrong to impose extra hypotheses!
References.
[HTT] J. Lurie, Higher topos theory. Annals of Mathematics Studies 170. Princeton University Press, 2009.
[Spaltenstein] N. Spaltenstein, Resolutions of unbounded complexes. Compos. Math. 65.2, p. 121-154 (1988).
[Stacks] The Stacks project.
