If not, is there an easy counterexample? Are there restrictions on $G$ and $H$ such that it is true? If yes, does anyone know a reference?

5$\begingroup$ That is not true. Already the group $\textbf{SL}_n$ is the total space of an etale $\mu_n$torsor over $\textbf{PGL}_n$ that is not split. $\endgroup$ – Jason Starr Mar 1 at 9:05

$\begingroup$ Presumably what's true is the the resulting object $P$ is a principal $Aut(P)$bundle and $Aut(P)$ is an extension of $G$ by $H$. $\endgroup$ – Denis Nardin Mar 3 at 11:00
I am just posting my comment as an answer. That is not true. One issue is that group schemes often admit isogenies where both the domain and the target are connected ("unbranched covers between groups"). Let $k$ be an algebraically closed field. Let $n$ be an integer relatively prime to the characteristic. Let $G$ be the commutative group $k$scheme $\mu_{n,k}$, the group of $n^{\text{th}}$ roots of unity. Let $H$ be the commutative group $k$scheme $\mathbb{G}_{m,k}$, the multiplicative group. Denote by $E$ a second copy of $\mathbb{G}_{m,k}$. The $n^{\text{th}}$ power map, $$E\to H, \ \ t\mapsto t^n,$$ is a $\mu_{n,k}$torsor. Of course $H$ is an $H$torsor over $\text{Spec}\ k$. Thus, the $k$scheme $E$ is a $G$torsor over an $H$torsor over $\text{Spec}\ k$. If $E$ were a $G\times H$torsor, then $E$ would be disconnected (since $G$ is disconnected).
There are some questions related to this that are interesting and come up in the literature. Does every algebraic group $H$ with finite fundamental group admit an isogeny of $k$group schemes $E\to H$ that is a "universal cover", i.e., the kernel $G$ equals the fundamental group? When the fundamental group is infinite, is there an isogeny of proschemes that plays the same role? Does every $G$torsor over an $H$torsor admit a structure of $E$torsor if it has geometrically connected fibers? In a different direction, for connected groups $G$ and $H$, is every $G$torsor over an $H$torsor over a base scheme $B$ isomorphic as a $B$scheme to a $G\times H$torsor over $B$? Is this true "locally on $B$"?

$\begingroup$ Hi Jason, would you mind explaining the "If $E$ were a $G \times H$torsor, then $E$ would be disconnected" part? We have that $E \to H$ is a $G$torsor but $G$ is disconnected. $\endgroup$ – user2831784 Mar 3 at 21:10

$\begingroup$ @user2831784 I am not certain that I understand your question. The ground field $k$ is algebraically closed. Thus, every $G\times H$ torsor over $\text{Spec}\ k$ is isomorphic to $G\times H$. The $k$scheme $G\times H$ surjects to $G$. Since $G$ is disconnected, also $G\times H$ is disconnected. Does this answer your question? $\endgroup$ – Jason Starr Mar 3 at 22:21

One can see this is true even without looking at algebraic geometry. Let $G\to K\to H$ be a central extension of finite groups that is not split (and, for the sake of it, take $G$ and $H$ abelian, but $K$ nonabelian). Then in pretty much any geometric setup one chooses, $H\to 1$ is a principal $H$bundle, where $1$ is the terminal object in your category of geometric objects. And $K\to H$ is a principal $G$bundle, but $K\to 1$ is not, I'm pretty sure. a principal $G\times H$ bundle, at least in a way compatible with the given structure. That is, one shouldn't be able to extend the given $G$action on $K$ to a principal $G\times H$action, and the $H$action arising from restricting the $G\times H$action won't cover the $H$action on $H$. Notice that we have a given basepoint in $K$, and so would be able to recover the group structure from the $G\times H$torsor structure.
If one is allowed some more geometry, then Jason's answer is spot on, and analogues of it hold for a differential geometric setup.

1$\begingroup$ I guess the simplest example is $G = H = {\mathbb Z}/2{\mathbb Z}$ and $K = {\mathbb Z}/4{\mathbb Z}$. $\endgroup$ – Michael Stoll Mar 3 at 13:19

$\begingroup$ Or, if one drops the central part (which isn't really relevant) $\mathbb{Z}/3 \to S_3 \to \mathbb{Z}/2$, so getting a nonabelian example. $\endgroup$ – David Roberts Mar 3 at 20:29