It is right, and not so obvious.
The question of whether or not a Markov process hits
particular sets is usually studied using the concept of capacity.
For a continuous time parameter Markov process taking values in
a general topological state space, this leads to non-trivial problems of measurability.
For instance, for a Borel $A$ there is no guarantee that the set $R(T,A)\in{\cal F}$
where $(\Omega,{\cal F},\Pr)$ is the probability space.
However, under suitable conditions, capacity theory can be used to show that $R(T,A)$ is universally measurable, and hence that $\Pr[R(T,A)]$ makes sense.
Let's assume that the state space and process are "nice";
say, the state space is a locally compact, separable metric space,
and the process has right continuous sample paths.
For fixed $T<\infty$, the formula $\phi(A)=\Pr[R(T,A)]$ defines a Choquet capacity on the
Borel sets $A$. Therefore, $$\phi(A)=\sup(\phi(K): K\subseteq A,\ K\mbox{ compact}).$$
For a compact $K$, define the stopping time $\tau(\omega):=\inf(t\geq 0: X_t(\omega)\in K)$.
Since the sample paths of $(X_t)$ are right continuous and $K$ is
closed, we have $R(T,K) = (X_{\tau\wedge T} \in K).$
Therefore,
$$\Pr[R(T,K)]\leq \mathbb{E}[I_A(X_{\tau\wedge T})]\leq \Pr[R(T,A)].$$
Taking the supremum over compact subsets of $A$ gives
$$ \Pr[R(T,A)]=\sup_{\tau}\ \mathbb{E}[I_A(X_{\tau\wedge T})],$$
which gives your desired result. Letting $T\to\infty$ gives the infinite version.
The result hinges on the fact that, as far as the process goes, the Borel set $A$ can be well
approximated from the inside by compact sets.
You can find more details in Chapter I, Section 10 of Blumenthal and Getoor's
Markov Processes and Potential Theory, or in Section 3.3 of Kai Lai Chung's Lectures from Markov Processes to Brownian Motion.
