I am working through the book "Foundations of Stochastic Inventory Theory".
One of the results in the book is Theorem 11.2. The background to this theorem is as follows.
Given finite state space $\mathcal{S}=\{1,2,...,S\}$, policy (which provides a mapping of what action to take in each state $s$ in teach time period $t$:) $\mathcal{\pi}=\{\pi_1,\pi_2,...,\pi_t,...\}$, and a value space $\mathbb{R}^S$. Then, to obtain period $t$'s value function, $v_t(\pi)$ (which is an $S$ vector of reals), we can solve the following recursion:
$$v_t(\pi)=H(\pi_t)v_{t+1}(\pi) $$
In plain language, this means that given an $S$ vector of period $t+1$'s values of being in different states, the value of starting in different states in period $t$ can be obtained by applying operator $H(\pi_t)$ to the next period's values. It is known that $H(\pi_t)$ is an affine map and is also a contracting operator. That is, given a suitable metric, for all $t$, $d(H(\pi_t)u,H(\pi_t)v)<d(u,v)$. As a result, it is known that a unique fixed point exists. Operator $H(\delta)v=H_{\delta}v=r_\delta+Q_\delta v$. Here, $r_\delta$ is an $S$-vector of immediate payoffs in the corresponding states. $Q_\delta=\alpha P_\delta$, where $0\leq \alpha<1$ can be thought of as a discount factor, and $P_\delta=[p_\delta^{ij}]$ is the $S\times S$ matrix of transitioning from state $i$ in one period to state $j$ in the next period in following decision rule prescribed by $\delta$.
Now, the theorem states:
The infinite horizon values exist for every policy: $v_t(\pi)$ exists and is finite for every $t$.
However, it is not clear to me how this can be established. What the fixed point result seems to state is that for arbitrary $\epsilon>0$, and an arbitrary value function $v$, there exists a sufficiently large index $M$ such that if we have $v_M:=v\in \mathbb{R}^S$, we can recursively obtain $v_{M-1}=H(\pi_{M-1})v_M$, and then sequentially iteratively obtain previous time period's value function until we get $v_0$ and this $v_0$ is going to be $\epsilon$-close to the fixed point in metric $d()$.
This theorem, however, seems to want one to prove the result in the other direction, beginning from a particular $t$ and then going to $t+1, t+2,...$. Any pointers would be appreciated.
