Let $P_{\mathbf{p}}$ be a $n \times n$ row-stochastic matrix whose entries are a function of a probability vector $\mathbf{p} \in \mathbf{R}_{> 0}^n$, $\sum_i p_i = 1$ and define the following mapping: $$ T(\mathbf{p}) = \mathbf{p} \lim_{m \to \infty} P_{\mathbf{p}}^m. $$ Note that $T(\mathbf{p})$ can be seen as the stationary distribution of the Markov chain with transition matrix $P_\mathbf{p}$. I would like to show that the Jacobian matrix of this mapping, that I define as $$ (*) \;\; T'_{ij} = \frac{\partial T_j}{\partial p_i}, $$ is also row-stochastic. So far, I managed to show that the Jacobian matrix of the (simpler) mapping: $$ \tilde{T}(\mathbf{p}) = \mathbf{p} \ P_{\mathbf{p}} $$ is row-stochastic.

Question: is there some way I can use this knowledge in my (bigger) problem above? Maybe some clever application of a chain rule for derivatives?

One approach I've been thinking about is an induction on $m$, but it seems quite tedious.

Edit on 06.10

Slightly changed my definition of Jacobian matrix $(*)$, using the transpose now.


There are a few issues here:

  1. The limit $Q_{\mathbf p}:=\lim_m P_{\mathbf p}^m$ may not exist in general.
  2. It is unclear if the limit is differentiable in $\mathbf p$, even when it exists and $P_{\mathbf p}$ is differentiable in $\mathbf p$.
  3. It is unclear how you define the Jacobian (matrix?). For it be row-stochastic, in your definition of the Jacobian matrix $i$ has to index the columns, and $j$ has to index the rows.

Once all these issues are cleared, that is, if the limit $Q_{\mathbf p}$ exists and is differentiable in $\mathbf p$, then one can deal with $\mathbf{p}Q_{\mathbf p}$ in exactly the same way as one deals with $\mathbf{p}P_{\mathbf p}$. Namely, writing $\mathbf p=[p_1,\dots,p_n]$, $Q_{\mathbf p}=(q_{i,j;\mathbf p})_{i,j=1}^n$, and $\mathbf{p}Q_{\mathbf p}=[t_{1;\mathbf p},\dots,t_{n;\mathbf p}]$, one has $$t_{j;\mathbf p}=\sum_i p_i q_{i,j;\mathbf p} $$ and hence $$\sum_j\frac{\partial t_{j;\mathbf p}}{\partial p_k} =\sum_j q_{k,j;\mathbf p} +\sum_j\sum_i p_i\frac{\partial q_{i,j;\mathbf p}}{\partial p_k} =1 +\sum_i p_i\frac{\partial}{\partial p_k}\sum_j q_{i,j;\mathbf p}=1, $$ since $\sum_j q_{i,j;\mathbf p}=1$.

However, in general one cannot guarantee that the elements of the Jacobian matrix of $\mathbf{p}Q_{\mathbf p}$ be all nonnegative, even if it is assumed that all the elements of the Jacobian matrix of $\mathbf{p}P_{\mathbf p}$ are positive. E.g., suppose that $\mathbf{p}=[s\ \;t]$ and $P=\begin{bmatrix} \frac{15}{16}-\frac{s}{4} & \frac{s}{4}+\frac{1}{16} \\ \frac{1}{8} & \frac{7}{8}\end{bmatrix}$ (I am dropping the subscript ${}_{\mathbf p}$). Then the stationary matrix $Q=\begin{bmatrix} \frac{2}{4 s+3} & \frac{4 s+1}{4 s+3} \\ \frac{2}{4 s+3} & \frac{4 s+1}{4 s+3}\end{bmatrix}$ and $\dfrac{\partial t_1}{\partial s}=\dfrac{6-8 t}{(4 s+3)^2}<0$ if $\frac34<t<1$, whereas all the elements of the Jacobian matrix $\begin{bmatrix}\frac{1}{16} (15-8 s) & \frac{1}{8} \\ \frac{1}{16} (8 s+1) & \frac{7}{8}\end{bmatrix}$ of $\mathbf{p}P_{\mathbf p}$ are positive.

  • $\begingroup$ Iosif, thanks a lot. What's missing is to show that the entries of the Jacobian matrix are all non-negative. Or is this trivial somehow? $\endgroup$
    – lum
    Jun 10 '15 at 8:33
  • $\begingroup$ As for your comments: 1) + 2) True. In my case the limit exists and is differentiable. 3) You are right, I changed my definition of the Jacobian matrix accordingly. Also, I guess you mean $\mathbf{p}Q_{\mathbf{p}} = [t_{1;\mathbf{p}}, \ldots, t_{n;\mathbf{p}}]$. $\endgroup$
    – lum
    Jun 10 '15 at 8:36
  • $\begingroup$ In my answer, I have fixed the typo concerning $\mathbf{p}Q_\mathbf{p}$ and added a comment about the (lack of) nonnegativity, in general. $\endgroup$ Jun 10 '15 at 15:52
  • $\begingroup$ But the Jacobian matrix of $\mathbf{p}P$ is row-stochastic by assumption. Can this help? $\endgroup$
    – lum
    Jun 10 '15 at 17:05
  • $\begingroup$ Reading your question, I had thought that you proved that the Jacobian matrix of pP is row-stochastic, instead of assuming it is so. Anyway, even this assumption does not help, as shown in the added, last paragraph of my answer. $\endgroup$ Jun 10 '15 at 20:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.