# Markov renewal process with failure?

I hope this question is not too elementary for this site, and that it contains a sufficient degree of detail.

I have a problem where I want to model sequences of variable length $\boldsymbol{e}_i = (e_{i1}, e_{i2}, \ldots, e_{iK_i})$. Here, $e_{it} = (X_{it}, T_{it})$ and there are three levels of randomness:

1. Length of the sequence ($K$). The sequence always ends with a sale event beyond which the process does not exist, and it is not just due to sampling.

2. The states at each element of the sequence ($X$). The states arise from a finite state space (12).

3. The interarrival times of the jumps between elements of the sequence ($T$). The time period between jumps are random.

Here are some of the models that I have evaluated and the features that match my case:

• Markov renewal processes/semi-Markov process: Here the change of state and the interarrival times of the jumps in the process are both random and modeled as a tuple. However, this does not incorporate the sale event as terminating the chain.

• Absorbing Markov chains: Here there is an absorbing state which culminates the process, but it seems to physically connote that the process continues. In any case, I have not been able to find a suitable absorbing semi-Markov chain.

There are other references to truncated and terminating Markov chains, but I have not followed up on those. Any ideas as to the right way to model and simulate these processes would be highly appreciated.

I am not sure where I should include an auxiliary failure process which governs when the process fails. If it helps anyone, I can add a description of the context, but all the features that I would like to model are present here.

-
Can you be more specific about what your problem with modelling is? I mean, as posed now, there seems to be nothing wrong with an absorbing Markov chain. Just terminate your sampling when you reach the absorbing state. – fedja Feb 7 '13 at 11:49
@fedja In that case, can you refer me to some applications of absorbing semi-Markov models? – tchakravarty Feb 7 '13 at 12:03