Your question comes down to knowing that if $G$ (your $G/Z(G)$) is a smooth connected affine extension of ${\rm{PGL}}_2$ by a finite group then the quotient map $q:G \rightarrow {\rm{PGL}}_2$ is either an isomorphism or the central extension by $\mu_2$. Observe that $\ker q$ is finite and normal in the connected $G$, so it is central (as you're in char. 0).
Such a $G$ must be reductive, even semisimple (definitions via vanishing of unipotent radical and radical, not via representation theory), since ${\rm{PGL}}_2$ is visibly semisimple. Thus, every maximal torus in $G$ is its own centralizer (!) and hence contains the central $\ker q$. These maximal tori are 1-dimensional, so $\ker q$ is a finite subgroup of a 1-dimensional torus and hence is $\mu_n$ ($n$th roots of unity) for some $n \ge 1$. Hence, for any maximal torus $T$ in $G$, $\ker q$ is exactly its unique subgroup $T[n]$ of order $n$ (namely, the group $\mu_n$ of $n$th roots of unity).
The image of $T$ in the isogenous quotient ${\rm{PGL}}_2$ is a maximal torus $D$ in the latter. By conjugacy of maximal tori in ${\rm{PGL}}_2$ and inspection for the diagonal torus, there is $w \in {\rm{PGL}}_2$ that normalizes $D$ with $w$-conjugation acting on $D$ via inversion. Since $T$ is the preimage of its image $D$ (as the central $\ker q$ is contained in all maximal tori of $G$), any $g \in G$ lifting $w$ must normalize the 1-dimensional $T$ with $g$-conjugation acting on $T$ via its unique nontrivial automorphism, namely via inversion. Ah, but the subgroup $\ker q = T[n]$ is central in $G$, so $g$-conjugation is trivial on this. In other words, inversion (on $T$) has trivial effect on $T[n]$, forcing $n \le 2$. Voila, so either $q$ is an isomorphism or it is a central extension of degree 2.
Now we can go a bit further and show that if $n = 2$ then $G = {\rm{SL}}_2$ (as a central extension of ${\rm{PGL}}_2$). Namely, consider the pullback $$\widetilde{G} = G \times_{{\rm{PGL}}_2} {\rm{SL}}_2,$$ or more specifically its identity component. This is an isogenous smooth connected affine cover of ${\rm{PGL}}_2$ that has degree at least 2 and yet dominates the degree-2 cover ${\rm{SL}}_2$. The preceding shows that its degree over ${\rm{PGL}}_2$ is at most 2, whence exactly 2, so its covering map onto ${\rm{SL}}_2$ is degree 1 (i.e., an isomorphism).