"Natural" action of a semidirect product of groups Suppose we have a group $N$ acting on a set $X$, and a group $Q$ acting on $Y$. Moreover, we are given $\phi: Q \to \mathrm{Aut}(N)$. 
My question is, is there a "natural" action on a set that can be constructed from $X$ and $Y$ (maybe $X \times Y$?) on which the semidirect product $N \rtimes_{\varphi} Q$ acts. 
For example, when $\phi$ is trivial (so we are considering direct product of groups), then the answer to this question is trivial (our set is $X \times Y$ and the action is coordinatewise). 
PS In general I am interested in a case when $X$ and $Y$ have more structure, but let us start with sets. 
 A: Suppose you define some equivalence relation on $X$ so that the quotient $\overline{X}$ has the property that $N\rtimes Q$ acts on $\overline{X}\times Y$ coordinatewise, with the action of $n$ on $[x]$ being $[nx]$ (here, $[x]$ is the equivalence class of $x$).
Writing ${}^qn$ for $\varphi(q)(n)$, and writing the elements of $N\rtimes Q$ as pairs $(n,q)$, we would need $[nx] = [{}^qnx]$ for all $q\in Q$ (with fixed $x$ and $n$). For $({}^qn,1) = (1,q)(n,1)(1,q^{-1})$, and looking at the actions on a pair $([x],y)$ readily gives the need for $[nx]=[{}^qnx]$. 
Conversely, if $\sim$ is any equivalence relation on $x$ that satisfies this property, then the action on $\overline{X}\times Y$ given by $(n,q)([x],y) = ([nx],y)$ is readily seen to be an action:
$$\begin{align*}
(n_1,q_1)\Bigl((n_2,q_2)([x],y)\Bigr) &= (n_1,q_1)([n_2x],q_2y)\\\
&= ([n_1(n_2x)],q_1(q_2y))\\\
\Bigl((n_1,q_1)(n_2,q_2)\Bigr)([x],y) &= (n_1{}^{q_1}n_2,q_1q_2)([x],y)\\\
&= ([n_1{}^{q_1}n_2 x],(q_1q_2)y).
\end{align*}$$
The assumption on $\sim$ gives that $[{}^{q_1}n_2 x] = [n_2x]$, and hence that $[n_1{}^{q_1}n_2 x] = [n_1n_2x]$.
If you let $\sim$ be the transitive closure of the relation with $x\sim x'$ if and only if there exists $x_0\in X$, $n\in N$, and $q\in Q$ with $nx_0 = x$, ${}^qnx_0 =x'$, then any equivalence relation that is coarser than $\sim$ will do. (I may have my "coarser" and "finer" mixed up here; I mean that the equivalence classes are unions of equivalence classes under $\sim$, so that if you think of it as if it were a "topology", you would have a coarser topology, with fewer open sets).
In particular, taking $\sim$ to be the total relation ($x\sim x'$ for all $x,x'\in X$), you get the natural action on $Y$ given by mapping to the quotient (well, technically an action on $\{\bullet\}\times Y$, but that is naturally isomorphic to an action on $Y$). If the action of $Q$ on $N$ is trivial, so that ${}^qn = n$, then $\sim$ is the identity relation, so $\overline{X}=X$ and you get the natural action on $X\times Y$. 
A: In general, $N$ (acting on itself) is a natural answer. Let us consider the example of the affine group of a vector space $V$: the affine group is the semi-direct product of $Aut(V)=Gl(V)$ and the additive group of $V$. This is "the" natural space where $Aff(V)$ acts.
