I have a question that is an extension of this one. My question is: Can we say that for every policy, there exists a deterministic policy in case of a finite-state, finite-action infinite-horizon discounted (where the discount factor is fixed)constrained MDP, where the costs and transition probabilities are known? (i.e. when there are multiple discounted costs with one cost in the objective and the rest with bounds in the constraints)
5
-
$\begingroup$ Since the answer depends on the assumptions (see the book of Puterman in one of the answers) please be more specific about your assumptions. $\endgroup$– Dieter KadelkaCommented Jan 13, 2021 at 14:17
-
$\begingroup$ Can you please clarify what do you mean by assumptions here. $\endgroup$– user812951Commented Jan 13, 2021 at 14:52
-
$\begingroup$ I am asking what would be the structure of the optimal policy when there are multiple discounted costs. Please let me know if I should add more to the question. $\endgroup$– user812951Commented Jan 13, 2021 at 14:59
-
$\begingroup$ The problem I have with your question simply is, that there are many versions of discounted MDP. The state space $S$ may be be finite, countable infinite or a measurable space. Similar for the action space, the restrictions (constraints), the reward/loss function, the discount factor may even depend on the actual state. Maybe that In the books of S.M.Ross: Introduction to stochastic dynamic programming (1983) or E.V. Denardo: Dynamic Programming (1982) you find the answer you seek for your specific model. $\endgroup$– Dieter KadelkaCommented Jan 13, 2021 at 15:09
-
$\begingroup$ Got it, Thank you so much for pointing out. $\endgroup$– user812951Commented Jan 13, 2021 at 15:34
Add a comment
|