A Markov chain $X$ with finite state space $\{1,2,\cdots,N\}$ is defined on a probability space $(\Omega, P, \mathcal{F})$ equiped with filtration $\{\mathcal{F}_t\}$. And we assume that we can reach all the states. We consider a process $\{R_t\}$ which is $\mathcal{F}_t$ supermartingale, therefore for any two stopping times $ T \geq S$, $E[R_T|\mathcal{F_t}] \leq E[R_S|\mathcal{F_t}]$ (due to optional stopping theorem). Now, let $g$ be a real valued bounded function of $\mathbb{R}$.

Intuitively, I seems obvious to me that:

$$\sup_{t \leq \tau \leq \infty} E[R_{\tau} g(X_{\tau})| \mathcal{F}_t] = \max_{k\in {1,\dots,N}}E[R_{\tau_k}g(X_{\tau_k})|\mathcal{F}]$$ where

$\tau_k= \inf\{s \geq t, X_s= k\}$.

I am having trouble showing it though, the 'math' behind my intuition is this : take the supremum over the partition $\cup_k \tau(\{\omega: X_{\tau} =k \})$ and then the supremum will become

$\max(\sup_{\tau( \{\omega: X_{\tau} =1 \})}E[R_\tau g(1)|\mathcal{F}_t], \cdots, \sup_{\tau( \{\omega: X_{\tau} =N \})} E[R_\tau g(N)|\mathcal{F}_t]) = \max_{k\in {1,..,N}}E[R_{\tau_k}g(X_{\tau_k})|\mathcal{F}]$

as $\sup_{\tau( \{\omega: X_{\tau} =k \})} E[R_\tau g(k)|\mathcal{F}_t]= E[R_{\tau_k}g(X_{\tau_k})|\mathcal{F}]$.

I am not sure about my first step though i.e the supremum over the union.

I appreciate any suggestion. Thanks.