It looks to me like under ZFC, $\mathbb{R}$-satisfiability is not (consistently) $(\omega_2,\omega_3)$-compact. To see this, we'll emulate your argument above for $\mathbb{R}_{\mathbb{Z}}$. So basically, we want a theory with constants $c_\eta$ for $\eta<\omega_2$, which says:
"$c_\eta$ codes a wellorder of $\omega$ (in ordertype $\geq\omega$)",
"if $\eta_0\neq\eta_1$ then $c_{\eta_0},c_{\eta_1}$ have distinct ordertypes".
It suffices to express these things in an appropriate fashion. We will in fact use a bunch more constants to help with this. We start with constants $\left<n_i\right>_{i<\omega}$, along with the statements:
So whenever all these formulas are included, $n_k$ must be interpreted by $k$. Thus, we can talk about the integers by referencing these constants. But we cannot quantify directly over the integers. But we can do this enough for our purposes, indirectly. For each tuple $\vec{a}=(a_0,\ldots,a_{k-1})$ of constants that we add, we add further new constants $t_{\vec{a}}$ and $t^{\mathrm{wit}}_{\vec{a}}$; $t_{\vec{a}}$ will code the $\Sigma_1^{\mathbb{N},\vec{a}}$ theory (by which I really mean that $\Sigma_1^{\mathbb{N},A_{\vec{a}}}$ theory, where $A_{\vec{a}}=(A_{a_0},\ldots,A_{a_{k-1}})$ denotes the tuple of sets of integers coded by $\vec{a}$, and the theory is the collection of all $\Sigma_1$ truths in integer parameters over the structure $(\mathbb{N},A_{a_0},\ldots,A_{a_{k-1}})$), and the (interpretation of the) constant $t^{\mathrm{wit}}_{\vec{a}}$ will code witnesses to the $\Sigma_1$ assertions in this theory. Because our constants are closed under this, we end up with constants which will code the $\Sigma_k^{\mathbb{N},\vec{a}}$ theory, for each $k<\omega$, via which we can make arithmetical statements, which will be very helpful.
So, for specificity, let's say that the real $x$ codes the set of rationals ${<x}$, and this then gives us a natural way of coding a set $A_x$ of integers with $x$. Fix also such a natural way of coding total functions $f_x:\omega\to\omega$ with $x$. (For example, take $f_x(0)=$ the floor of $x$, and if $f_x(0)=i$, then break the interval $[i,i+1)$ into the sub-intervals $I_0=[i+\frac{1}{2})$, $I_1=[i+\frac{1}{2},i+\frac{3}{4})$, etc, and $f_x(1)=$ the $k$ such that $x\in I_k$, and then break $I_k$ up in this manner to define $f_x(2)$, etc.)
Note that for each specific tuple $\vec{n}$ of integers, we can express $\Sigma_0^{\mathbb{N},\vec{a}}(\{\vec{n}\})$ statements with formulas in our theory, by converting them into quantifier-free statements (i.e. convert "$\exists m<n_0$"
as a disjunction of $n_0$-many formulas, etc). Fix an enumeration $\left<\psi_i\right>_{i<\omega}$ of all $\Sigma_0$ formulas in the language of $\mathbb{N}$ plus a predicate (implicit below, interperted as $A_\vec{a}$). We include the following statements in our theory:
"If the formula `$\exists k\psi_i(k,\vec{n})$' is in $A_{t_{\vec{a}}}$,
and $f_{t_{\vec{a}}^{\mathrm{wit}}}(i)=k'$, as witnessed by calculation $c$,
then $\psi_i(k',\vec{n})$" (here $i,\vec{n},k',c$ are arbitrary (tuples of) integers, and we have some fixed coding of "calculations" by integers; note that "$\psi_i(k',\vec{n})$", for these particular integers $k',\vec{n}$, is expressible as noted above).
"If the formula `$\exists k\psi_i(k,\vec{n})$' is not in $A_{t_{\vec{a}}}$, then $\neg\psi_i(k',\vec{n})$" (here $i,k',\vec{n}$ are arbitrary integers).
If we have all these formulas present in the theory, then $t_{\vec{a}}$ must be interpreted by (some real coding) the true $\Sigma_1^{\mathbb{N},\vec{a}}$-theory. Thus (and as mentioned above) we can make arithmetical statements about the (sets coded by the) constants.
We now want to express "$c_\eta$ codes a wellorder of $\omega$ (of ordertype $\geq\omega$)". We can do this by saying it codes a linear order which is comparable with all countable ordinals. Toward this, fix surjections $f:\omega\to\alpha$, for each $\alpha\in[\omega,\omega_1)$. We introduce new constants $d_\alpha$ for $\alpha\in[\omega,\omega_1)$, and, along with $c_\eta$, new constants $\pi_{\eta\alpha}$ for $\alpha\in[\omega,\omega_1)$. We want $d_\alpha$ to code a wellorder of ordertype $\alpha$, via $f_\alpha$, and $\pi_{\eta\alpha}$ to be an isomorphism comparing $c_\eta$ with $d_\alpha$. So add the following statements which achieve this:
"$(m,n)\in A_{d_\alpha}$" (for integers $m,n$ and $\omega\leq\alpha<\omega_1$
such that $f_\alpha(m)<f_\alpha(n)$),
"$(m,n)\notin A_{d_\alpha}$" (for $m,n,\alpha$ as above but with $f_\alpha(m)\geq f_\alpha(n)$",
"$c_{\eta}$ codes a linear order $<_\eta$ of $\omega$" (for all $\eta$),
"$\pi_{\eta\alpha}$ codes an isomorphism, either from $c_\eta$ onto an initial segment of $d_\alpha$, or from an initial segment of $c_\eta$ onto $d_\alpha$" (all $\eta,\alpha$).
Note these are each arithmetic statements about the relevant parameters. Finally, we want to say that the ordertypes of the $c_\eta$ are pairwise distinct. This is similar.
We use further constants $\sigma_{\eta\beta}$ for $\eta<\beta<\omega_2$, and say:
- "$\sigma_{\eta\beta}$ codes an isomorphism, either from $<_\eta$ onto a proper segment of $<_\beta$, or from a proper segment of $<_\eta$ onto $<_\beta$" (for each $\eta<\beta<\omega_2$).
Now as in your $\mathbb{R}_{\mathbb{Z}}$ example, every sub-theory of size ${<\omega_2}$ is $\mathbb{R}$-satisfiable, but the whole thing is not, because any realization is really "correct", i.e. the constants get the desired kinds of meanings.
Update: Re the further question in the comments on $(\omega_3,\omega_4)$: Assume ($\dagger$) ZFC + for every real $x$, $x^\#$ exists, and $u_2=\omega_2$ (where $u_\alpha$ is the $\alpha$th uniform indiscernible). Then $\mathbb{R}$-satisfiability is not $(\omega_3,\omega_4)$-compact.
Remark: Woodin has shown that ($\dagger$) is consistent relative to ZFC + a Woodin cardinal + a measurable above it. Moreover, starting with a model $V\models(\dagger)$, we can force over $V$ by adding a sequence $G$ of Cohen reals, hence arranging $2^\omega$ as large as we want, and preserve ($\dagger$). (For $u_2$ can only increase by forcing in general, and $u_2\leq\omega_2$ in general. But $\omega_n^{V[G]}=\omega_n$, and it follows that $V[G]\models u_2=\omega_2$. So it suffices to see that all reals still have sharps. But every real in $V[G]$ is added by a single Cohen forcing, and Cohen forcing preserves the existence of sharps for all reals.) So it's not that $(\omega_3,\omega_4)$-compactness is necessarily failing due to small continuum.
So assume ($\dagger$). The failure of $(\omega_3,\omega_4)$-compactness is arranged along the lines of the preceding argument. Define the equivalence relation $=^*$ on the reals by $x=^*y$ iff $\kappa_x=\kappa_y$, where $\kappa_x$ is the least $x$-indiscernible $\kappa$ such that $\kappa>\omega_1$. By ($\dagger$), $\{\kappa_x\bigm|x\in\mathbb{R}\}$ is cofinal in $\omega_2$, so there are exactly $\omega_2$-many equivalence classes. So let $\left<c_\eta\right>_{\eta<\omega_3}$ be some constants. It suffices to see we can express the statements
- "$c_\eta\neq^* c_\beta$",
for each $\eta<\beta<\omega_3$ (with the help of further constants). Given any constant $x$ we introduce, we will also introduce $s(x)$, which will be interpreted as $x^\#$. For all constants $s,t$, we will also introduce a constant $p(s,t)$ for $s\oplus t$. As before, we also introduce the $n_k$'s and close under the $\Sigma_1$-theories, so that we can make arithmetic statements. Now the main issue is to ensure that the interpretation of $s(x)$ is really $x^\#$: given this, the assertion "$c_\eta\neq^* c_\beta$" is equivalent to "$L[s(p(c_\eta,c_\beta))]\models c_\eta\neq^* c_\beta$" (where we define $=^*$ in this "model" $M$ as in $V$, except that we use $\omega_1^M$ instead of $\omega_1^V$), which will say that $L[(c_\eta\oplus c_\beta)^\#]\models$"$c_\eta\neq^* c_\beta$"; note that we can check this by just checking that $s(s(p(c_\eta\oplus c_\beta)))$ contains the right statement.
So we want to ensure that $s_x$ really gets interpreted as $x^\#$. We add further constants: Add $d_\alpha$ for $\omega\leq\alpha<\omega_1$ as before (coding a wellorder $<_\alpha$ of $\omega$ of ordertype $\alpha$), and $s_{x,\alpha}$ (to code the model of the form "$L_\gamma[x]$" (as of yet possibly illfounded) one gets generated from an $\alpha$-sequence of "$x$-indiscernibles" as determined by $s_x$), for $\alpha<\omega_1$, and $\pi_{x,\alpha,\beta}$, to code an isomorphism which compares the ordinals of $s_{x,\alpha}$ with $\beta$, for each $\alpha,\beta<\omega_1$. We add formulas asserting:
"$s_{x}$ is a pre-$x$-sharp" (meaning that $s_x$ has the right syntactic properties for a sharp, but ignoring iterability / wellfoundedness of the models generated as above),
"$(m,n)\in A_{d_\alpha}$" or "$(m,n)\notin A_{d_\alpha}$" (as before),
"$s_{x,\alpha}$ codes the term model generated by an $\alpha$-sequence of indiscernibles modelling the theory given by $s_x$ (and using $<_\alpha$ to determine the indiscernible sequence)",
"$\pi_{x,\alpha,\beta}$ codes either (i) an isomorphism from the ordinals of $s_{x,\alpha}$ onto an initial segment of $<_\beta$, or (ii) an isomorphism from an initial segment of the ordinals of $s_{x,\alpha}$ onto $<_\beta$".
These are all arithmetic statements about these constants, so we can make them. And they together ensure the correctness of $s_x$, because it ensures that the model $L_\gamma[x]$ mentioned above (which is countable) is wellfounded.
As mentioned above, since there are $\omega_2$ distinct classes with respect to $=^*$, but not $\omega_3$, this gives a failure of $(\omega_2,\omega_3)$-compactness for $\mathbb{R}$-satisfiability.
Thus, what if we work in $V=L[G]$ where $G$ adds $\geq\omega_4^L$ Cohen reals to $L$? Does $L[G]\models$"$\mathbb{R}$-satisfiability is $(\omega_3,\omega_4)$-compact"?
And however, I think $(\omega_4,\omega_5)$ will be more subtle.
Re $(\kappa,\kappa^{++})$-compactness for $\mathbb{R}$-satisfiability, don't we get that from $\kappa^+$-supercompactness of $\kappa$, like with the measurability giving $(\kappa,\kappa^+)$? (And likewise for $(\kappa,\lambda)$-compactness, if $\kappa$ is $<\lambda$-supercompact.) I.e. if $T$ is a theory of size $\gamma\in[\omega,\lambda)$, and all subtheories of size $<\kappa$ are $\mathbb{R}$-satisfiable, and $j:V\to M$ is elementary with $\gamma<j(\kappa)$ and ${^\gamma}M\subseteq M$, then $M$ thinks "all subsets of $j(T)$ of size ${<j(\kappa)}$ are $\mathbb{R}$-satisfiable". But $j``T\in M$ and $j``T$ has size $\gamma<j(\kappa)$ in $M$, so $j``T$ has an $\mathbb{R}$-model $\mathbb{R}^+\in M$, but note $\mathbb{R}^+\models T$. (Of course this actually has nothing to do with $\mathbb{R}$; it could have been any structure of size ${<\kappa}$.)
Update 2: Although it cannot be $(\omega_2,\omega_3)$-compact,
it turns out that, relative to ZFC + a weakly compact,
$\mathcal{R}$-satisfiability can be $(\kappa,\kappa^+)$-compact with uncountable $\kappa$,
even with large continuum, i.e. $2^{\aleph_0}>\kappa$. (On the other hand, in connection with the foregoing argument, and this question https://math.stackexchange.com/questions/4166734/does-mathsfzfc-prove-that-the-field-of-real-numbers-has-one-of-these-compac/4172264#4172264
and its answers,
a reasonable question was whether every uncountable cardinal $\kappa$
such that $\mathcal{R}$-satisfiability is $(\kappa,\kappa^+)$-compact, has to be weakly compact.) It turns out that the most obvious candidate for a (counter)example works: Suppose $\kappa$ is weakly compact in $L$. Let $G$ be $L$-generic for adding a $\kappa^+$-sequence of Cohen reals with finite support. Then in $L[G]$, $\mathcal{R}$-satisfiability is $(\kappa,\kappa^+)$-compact, but $2^{\aleph_0}=\kappa^+$, so $\kappa$ is not weakly compact.
Proof: Work in $V=L[G]$. Let $T$ be a relevant theory of size $\kappa$,
all of whose size ${<\kappa}$ sub-theories are $\mathcal{R}$-satisfiable. We want to see $T$ is also. Note that there is $\alpha<\kappa^+$ such that $T\in L_{\alpha}[G\upharpoonright\alpha]$, and so by rearranging $G$ a little, we may assume that $T\in L_\alpha[G\upharpoonright\kappa]$ for some $\alpha<\kappa^+$. Let $X\preccurlyeq L_{\kappa^{++}}$ with $\beta=X\cap\kappa^+$ transitive and $\alpha<\beta<\kappa^+$ and $|X|=\kappa$. Let $L_\gamma$ be the transitive collapse of $X$. Using the weak compactness of $\kappa$ in $L$,
let $\xi<\kappa^+$ and $j:L_{\gamma}\to L_\xi$ be elementary with $\mathrm{crit}(j)=\kappa$. So $\beta<j(\kappa)<j(\beta)=j(\kappa)^{+L_\xi}<\xi$. Note that $G\upharpoonright j(\kappa)$ is $L_\xi$-generic and extends $G\upharpoonright\kappa$, and we can extend $j$ elementarily to $j^+:L_\gamma[G\upharpoonright\kappa]\to L_\xi[G\upharpoonright j(\kappa)]$.
Now by the homogeneity of the forcing, $L_{\kappa^{++}}[G\upharpoonright\kappa]\models$"$\mathbb{P}$ forces that $T$ is ${<\kappa}$-$\mathcal{R}$-satisfiable", where $\mathbb{P}$ adds $\kappa^+$-many Cohen reals with finite support (and
the "$\mathcal{R}$" refers to the $\mathcal{R}$ of the $\mathbb{P}$-extension). So $L_\gamma[G\upharpoonright\kappa]$ models the same, but with $\mathbb{P}$ replaced by $\mathbb{P}\upharpoonright\beta$.
So $L_\xi[G\upharpoonright j(\kappa)]$ models the same regarding $j(T)$ and $j(\kappa)$ and $\mathbb{P}\upharpoonright j(\beta)$.
We have $j(T)\upharpoonright\kappa=T$, so $L_\xi[G\upharpoonright j(\kappa)]\models$"$\mathbb{P}\upharpoonright j(\kappa)$ forces that $T$ is $\mathcal{R}$-satisfiable". So $L_\xi[G\upharpoonright j(\beta)]\models$"$T$ is $\mathcal{R}$-satisfiable".
Let $\pi:T\to\mathbb{R}^{L_\xi[G\upharpoonright j(\beta)]}$ be an $\mathcal{R}$-realization of $T$ in $L_\xi[G\upharpoonright j(\beta)]$.
So $\pi\in L[G]$, and so it suffices to see that $\pi$ is also an $\mathcal{R}$-realization of $T$ in $L[G]$. For this, it suffices to see that $\mathcal{R}^{L_\xi[G\upharpoonright j(\beta)]}\preccurlyeq\mathcal{R}^{L[G]}$. But the theory of real closed fields has an algorithm for quantifier elimination, which both $L[G]$ and $L_\xi[G\upharpoonright j(\beta)]$ agree about, and since $\mathcal{R}^{L_\xi[G\upharpoonright j(\beta)]}$ is a sub-field (real closed) of $\mathcal{R}^{L[G]}$, this fact yields the desired elementarity.
A straightforward variant of this also works in $L[G]$ for any sequence $G$ of Cohen reals of length $\geq\kappa^+$.
