If I have a process that transitions between states with some set, unknown probability, I can sample to find the transition probability. This probability is a sample average, with a well understood sample distribution.

I can now use this probability to construct my markov chain, but if I simply use the sample average, I have an unbiased estimate of the input, but I don't know how the uncertainty propagates. For instance, if my transition probabilities make it look like this is an absorbing state, but in fact it is only close, I can significantly mis-specify my solution.

I can simulate it, but I think there should be some theoretical work on this, I just don't know where. I am looking for citations, or at least the terms I need to use. I assume some literature exists on this, but I cannot find what or where, because all the terms I search for (sample distribution, etc.) are used differently than I need, referring to the outputs, not the inputs...