I am reading papers about I-POMDP. I cant understand the finitely nested I-POMDPs given in these papers.
The belief update of the algorithm has a problem that agents' belief updates mutually depend on each others. For instance, the update functions for a system containing agent i and agent j are list as follows. \begin{equation} b_i^t(is_i^t) = \tau_{m_j^{t-1}} (b_i^{t-1}(is_i^{t-1});o_i^t,a_i^{t-1}) = \beta \sum_{is_i^{t-1} \vert \hat{\theta}_j^{t-1} = \hat{\theta}_j^t} {b_i^{t-1}(is_i^{t-1}) \sum_{a_j^{t-1}}{P(a_j^{t-1}\vert m_j^{t-1})P(o_i^t \vert s^t,a^{t-1})P(s^t \vert a^{t-1},s^{t-1})\sum_{o_j^t}{\delta_K(\tau_{m_i^{t-1}}(b_j^{t-1} ; o_j^t , a_j^{t-1})-b_j^t) P(o_j^t \vert s^t , a^{t-1})}}} \end{equation} \begin{equation} b_j^t(is_j^t) = \tau_{m_i^{t-1}} (b_j^{t-1}(is_j^{t-1});o_j^t,a_j^{t-1}) = \beta \sum_{is_j^{t-1} \vert \hat{\theta}_i^{t-1} = \hat{\theta}_i^t} {b_j^{t-1}(is_j^{t-1}) \sum_{a_i^{t-1}}{P(a_i^{t-1}\vert m_i^{t-1})P(o_j^t \vert s^t,a^{t-1})P(s^t \vert a^{t-1},s^{t-1})\sum_{o_i^t}{\delta_K(\tau_{m_j^{t-1}}(b_i^{t-1} ; o_i^t , a_i^{t-1})-b_i^t) P(o_i^t \vert s^t , a^{t-1})}}} \end{equation} As you can see above, calculation of $\tau_{m_j^{t-1}} (b_i^{t-1}(is_i^{t-1});o_i^t,a_i^{t-1})$ and $\tau_{m_i^{t-1}} (b_j^{t-1}(is_j^{t-1});o_j^t,a_j^{t-1})$ depend on the outcome of each other.
The papers discussing this topic give a solution called finitely nested I-POMDPs. The solution update the belief with subintentional model first for level 0. The belief update is as follows. \begin{equation} b_{i,0}^t(is_{i,0}^t) = b^t_{i,0}(s^t) = \beta \sum_{is_i^{t-1} \vert \hat{\theta}_j^{t-1} = \hat{\theta}_j^t} {b_{i,0}^{t-1}(is_{i,0}^{t-1}) \sum_{a_j^{t-1}}{P(a_j^{t-1}\vert m_j^{t-1})P(o_i^t \vert s^t,a^{t-1})P(s^t \vert a^{t-1},s^{t-1})\sum_{o_j^t}{\delta_K(\text{APPEND}(h_j^{t-1},o_j^t),h_j^t) P(o_j^t \vert s^t , a^{t-1})}}} \end{equation} \begin{equation} b_{j,0}^t(is_{j,0}^t) = b^t_{j,0}(s^t) = \beta \sum_{is_j^{t-1} \vert \hat{\theta}_i^{t-1} = \hat{\theta}_i^t} {b_{j,0}^{t-1}(is_{j,0}^{t-1}) \sum_{a_i^{t-1}}{P(a_i^{t-1}\vert m_i^{t-1})P(o_j^t \vert s^t,a^{t-1})P(s^t \vert a^{t-1},s^{t-1})\sum_{o_i^t}{\delta_K(\text{APPEND}(h_i^{t-1},o_i^t),h_i^t) P(o_i^t \vert s^t , a^{t-1})}}} \end{equation} Seemingly, the two update functions dont depend on each other now. But how could I do for level 1? I know I should solve the following updates for level 1. \begin{equation} b_{i,1}^t(is_{i,1}^t) = b_{i,1}(s^t, \langle b_{j,0}^t , \hat{m}_j^t \rangle) \end{equation} \begin{equation} b_{j,1}^t(is_{j,1}^t) = b_{j,1}(s^t, \langle b_{i,0}^t , \hat{m}_i^t \rangle) \end{equation} How to do the update on level 1 to avoid mututal dependence? I can't understand the solution from the papers. Thanks for any hints.