Timeline for Proof of Bellman optimality equation for finite Markov Decision Processes
Current License: CC BY-SA 4.0
5 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Sep 23, 2023 at 19:07 | comment | added | piero | @Sotiris expectation in $q_{\pi}$ as you pointed is not even defined | |
| Jan 27, 2019 at 19:44 | comment | added | Sotiris | Searching online @hardhu, I also found this answer on CrossValidaded, which is somewhat more explanatory: stats.stackexchange.com/questions/347268/… | |
| Jan 27, 2019 at 19:42 | comment | added | Sotiris | The equality @hardu is due to the fact that the policy $\pi_*'(a|s) = [a=a_s]$ is optimal by definition, given the fact that $a_s \in \text{arg}\,\max\limits_{a \in A}q_{\pi_*}(s,a) = \text{arg}\,\max\limits_{a \in A}q_{*}(s,a)$. Note that $q_{*}(s,a)$ is policy free and thus the choice $\pi_*'(a|s) = [a=a_s]$ represents the best choice that can be made, when in state $s$ (or one of the best, in case that there are many global maxima of $q_{\pi_*}(s,a)$ with respect to $a$). | |
| Jan 27, 2019 at 11:47 | comment | added | hardhu | See my answer for my doubts about yours. | |
| Jan 26, 2019 at 20:17 | history | answered | Sotiris | CC BY-SA 4.0 |