Consider the following extension of the notion of convexity of continuous functions to functions defined over $\mathcal{I} = \{0,\ldots,n \}$ — that is, to vectors: $f$ is convex if for any $\alpha \in [0,1]$ and for all $k\in \{ \ell \in \mathcal{I}: \ell \leq \alpha i+ (1-\alpha)j\}$ it holds that:

$$\alpha f_i + (1-\alpha)f_j \geq f_{k}.$$

Consider a Markov chain with right stochastic $n\times n$ transition matrix $P$ over the state space indexed by $\mathcal{I}$.

Consider the convex vector $g_i = i^2$ , I would like to know whether there is any condition — or whether it is even plausible that there might be some; I suspect not — to be placed on the stochastic matrix $P$ to ensure that

$$f := (I_n - \delta P)^{-1} g$$

is convex or not. Here $0 < \delta < 1$ and $I_n$ is the $n \times n$ identity matrix.

The motivation: $g$ is a cost incurred for each sojourn of the chain in state $i$, $f$ is the discounted expected cost of the chain when the future is discounted by $\delta$

A related problem is when instead one considers a similar chain but the $0-th$ state is taken to be absorbing and want to compute an approximation of the vector of expected absorption times given by

$$h := (I_{n-1} - \delta [P]_{0})^{-1} \boldsymbol{1}_{n-1}.$$

where $[P]_{0}$ is the sub-stochastic matrix obtained by deleting the $0-th$ column and row of $P$.

If this helps in any way, $P$ has a tridiagonal structure meaning that the Markov chain it represents is a nearest neighbor random walk on the line.