I assume that $(a_n)_{n \ge 1}$ are random variables taking values on a finite subset $B$, and that $\nu_l(b) \le P[a_n = b|a_1,\ldots,a_{n-1}] \le \nu_u(B)$ almost surely for every $n \ge 1$ and $b \in B$.
If yes, then for each $b \in B$, the formula $$M_n(b) := \sum_{k=1}^n\frac{1}{k}\big(1_{[a_k=b]}-P[a_k = b|a_1,\ldots,a_{k-1}]\big)$$ defines a square-integrable martingale. This martingale has orthogonal increments and is bounded in $L^2(P)$, since $$E\Big[\frac{1}{k^2}\big(1_{[a_k=b]}-P[a_k = b|a_1,\ldots,a_{k-1}]\big)^2\Big] \le \frac{1}{4k^2}.$$ Hence it converges almost surely and in $L^2$.
We deduce that the averages
$$\frac{S_n(b)}{n} := \frac{1}{n}\sum_{k=1}^n \big(1_{[a_k=b]}-P[a_k = b|a_1,\ldots,a_{k-1}]\big)$$
converge almost surely to $0$, by Cesàro lemma since
$$\frac{S_n(b)}{n} = \frac{1}{n}\sum_{k=1}^n k(M_k(b)-M_{k-1}(b))
= \frac{1}{n}\Big(nM_n(b) - \sum_{k=0}^{n-1}M_k(b)\Big).$$
As a result, the averages $\frac{S_n(b)}{n}$ and $\frac{1}{n}\sum_{k=1}^n P[a_k = b|a_1,\ldots,a_{k-1}]$ have the same limit points as $n \to +\infty$, which belong to $[\nu_l(b),\nu_u(b)]$.
ADDENDUM (answers to the questions added by the OP)
Step 1. $|M_n(b)| \le \sum_{k=1}^n 1/k$. Therefore $M_n(b)$ is in $L^2(P)$.
Step 2. On $L^2(\Omega,\mathcal{A},P)$, the conditional expectation $E[\cdot|\mathcal{F_n}]$ coincides with the orthogonal projection on $L^2(\Omega,\mathcal{F_n},P)$. Hence $M_{n+1}(b)-M_n(b)$ is orthogonal to $L^2(\Omega,\mathcal{F_n},P)$, therefore to $M_0(b),\ldots,M_n(b)$.
Step 3. Do not confuse $E[M_n^2]$ finite for every $n$ and $E[M_n^2]$ bounded independently on $n$. The last statement follows from Pythagore equality (write $N_n$ as the sum of the pairwise orthogonal random variables $M_1-M_0,\ldots,M_n-M_{n-1}$) and from the convergence of the series $\sum_k 1/k^2$.
Step 4. The theorem applied here is the martingale convergence theorem, for martingales which are bounded in $L^2(P)$. Convergence in $L^2(P)$ can also be proved simply bu using Cauchy lemma and Pythagore theorem, thanks to the pairwise orthogonalality of the random variables $M_n-M_{n-1}$ and the convergence of the series $\sum_k 1/k^2$.
Step 5. No question on this step.
Step 6. Two sequences $(u_n)$ and $(v_n)$ of real numbers whose difference converges to $0$ have the same limit points: remind that the limit points are the limit of convergent subsequences. Because of the assumption $u_n-v_n \to 0$, for every increasing map $\phi$ from $\mathbb{N}$ to $\mathbb{N}$, and every real number $\ell$, $u_{\phi(n)} \to \ell$ if and only if $v_{\phi(n)} \to \ell$.