Timeline for Existence of infinite horizon values in dynamic programming
Current License: CC BY-SA 4.0
9 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Feb 1, 2021 at 7:01 | comment | added | Michael Greinecker | The part of the result you mentioned in your question is relatively straightforward and amounts to calculating a slightly uglier version of a geometric series. The reward function is bounded and rewards are discounted. Writing down the formal details of the reward process induced by a policy is probably the hardest part in showing that it's expectation is finite. The part that requires a fixed-point argument is showing that the value of the policy satisfies the given recursion. | |
| Feb 1, 2021 at 6:06 | comment | added | Tryer | @MichaelGreinecker There is a separate operator $A$, which is defined as $Av:=\sup_{\delta \in \Delta}H_\delta v$. This theorem, however, does not talk about an optimal policy or $A$. It talks about any policy even if it is suboptimal. You can see the precise statement of the theorem at imgur.com/XGL8OAv | |
| Feb 1, 2021 at 4:39 | comment | added | Michael Greinecker | Are you sure $H$ is correctly written down? Usually, this should involve a maximization part. The fixed-point gives you then the optimal-value function and you can use it to find optimal policies.You can take a look at the Wiki entry for the Bellman equation. | |
| Feb 1, 2021 at 1:23 | comment | added | Tryer | @MichaelGreinecker Since you focus on game theory, I can add that Porteus specifies in his book that this theorem was stated/proven by Shapley in his 1953 paper on "Stochastic Games". He further states that he himself proved this in Porteus' 1982 paper "Conditions for characterizing the structure of optimal strategies in infinite horizon dynamic programs" that appearead in JOTA. I have both these paper, but this theorem does not seem to be stated / proven exactly in these works in my reading. | |
| Feb 1, 2021 at 1:19 | comment | added | Tryer | @MichaelGreinecker I described operator $H$ in greater clarify to specify the discount rate and the payoffs. | |
| Feb 1, 2021 at 1:18 | history | edited | Tryer | CC BY-SA 4.0 |
added 416 characters in body
|
| Jan 31, 2021 at 19:16 | comment | added | Michael Greinecker | There seems to be a lot missing. Is there a discount rate? How are payoffs determined? | |
| Jan 31, 2021 at 12:35 | review | First posts | |||
| Jan 31, 2021 at 13:19 | |||||
| Jan 31, 2021 at 12:32 | history | asked | Tryer | CC BY-SA 4.0 |