The wikipedia page cited in the question provides most of the answer: to get your operator compute
\begin{equation}
\lim_{\delta \rightarrow 0} \frac{ {\mathbb E}[f(X_\delta)] -f(x)}{\delta}
\end{equation}
The difference between your problem and the case covered in the wikipedia article is that $f$ in the above display is a function of $x$ only. However, your problem has an additional state variable (the binary variable that takes one of the values $1$ or $2$ depending on $\alpha$). So, the correct limit to study is:
\begin{equation}
\lim_{\delta \rightarrow 0} \frac{ {\mathbb E}[f(X_\delta,\alpha_t)] -f(x,1)}{\delta}.
\end{equation}
Thus, you don't have one function, but two functions $f(x,1)$ and $f(x,2)$ and two PDEs that these functions satisfy.
It is implicitly assumed that $\tau$ is independent of the dynamics of $X$ before $\tau$. Furthermore, before $\tau$ the dynamics of $X$ are governed by the first SDE given in the question. One can use these to write the above expectation in two pieces: one piece over the set $\{\delta < \tau\}$ the other over $\{\delta < \tau\}^c$. Once this is done, the usual use of Ito's formula gives:
$$
L_1 f(x,1)=-\lambda f(x,2)~~~ (*)
$$
and
$$
L_2 f(x,2) = 0.
$$
where
$$
L_i = a(b_i - x) \frac{\partial}{\partial x} + \frac{1}{2} \sigma^2\frac{\partial^2}{\partial x^2}
$$
Further details:
\begin{align*}
{\mathbb E}[ f(X_\delta,\alpha_\delta) ]&=
{\mathbb E}[ f(X_\delta,\alpha_\delta) 1_{\{ \tau > \delta\}} ] +
{\mathbb E}[ f(X_\delta,\alpha_\delta) 1_{\{ \tau \le \delta\} }]\\
&\approx (1-\lambda \delta){\mathbb E}[ f(X^1_\delta,\alpha_\delta)] +
\lambda \delta f(x,2),
\end{align*}
where $X^1$ is a process that is independent of $\tau$ with dynamics determined by $L_1$.
Here you use several things: 1) $P( \tau < \delta) \approx \delta \lambda$ 2) if a jump occurs before $\delta$, you can ignore what happens between $\tau$ and $\delta$ (the contribution of this part is in the order of $\delta^2$ and when divided by $\delta$ and
$\delta$ is let go to $0$, it disappears).
To get (*) from the previous display: use Ito's formula on the first expectation, subtract $f(x,1)$, divide by $\delta$ and let $\delta \rightarrow 0$. $f(x,2)$ is a function of what happens after $\tau$; after $\tau$ the stochastic process is a simple diffusion with generator $L_2$: this is why (**) holds.
