# Markov Chains based on sampled transition probabilities [closed]

If I have a process that transitions between states with some set, unknown probability, I can sample to find the transition probability. This probability is a sample average, with a well understood sample distribution.

I can now use this probability to construct my markov chain, but if I simply use the sample average, I have an unbiased estimate of the input, but I don't know how the uncertainty propagates. For instance, if my transition probabilities make it look like this is an absorbing state, but in fact it is only close, I can significantly mis-specify my solution.

I can simulate it, but I think there should be some theoretical work on this, I just don't know where. I am looking for citations, or at least the terms I need to use. I assume some literature exists on this, but I cannot find what or where, because all the terms I search for (sample distribution, etc.) are used differently than I need, referring to the outputs, not the inputs...

-

## closed as unclear what you're asking by Benoît Kloeckner, David White, j.c., Andrey Rekalo, Chris GodsilOct 25 '13 at 18:04

Please clarify your specific problem or add additional details to highlight exactly what you need. As it's currently written, it’s hard to tell exactly what you're asking. See the How to Ask page for help clarifying this question.If this question can be reworded to fit the rules in the help center, please edit the question.

I do not see a clear question here. Could you highlight what you are after precisely? –  Benoît Kloeckner Oct 25 '13 at 16:39

The problem is that even though you may obtain a unbiased estimate $\hat{S}$ of the stochastic matrix $S$, $\hat{S}^k$ is not an unbiased estimate of $S^k$, the k-steps transition matrix.
You can then sample whole chains, or even marginals within the chain, but every single chain will unfortunately represent a point hypothesis over $S$.