If we think about classical modular forms, one typically works on SL(2). One could also work on PSL(2), but one would like to write down congruence subgroups in terms of matrices, so one often phrases things for SL(2). Moreover, if one wants to deal with nebentypus, one is forced to work with SL(2).
However, if we want to study things representation theoretically, one usually passes to automorphic representations on GL(2), rather than SL(2). One loses nothing by doing so, and the representation theory is somewhat easier. E.g., there is one archimedean representation which is discrete series of weight $k$ for GL(2), rather than one of weight $k$ and one of weight $-k$ for SL(2). Globally we have strong multiplicity 1 for GL(2), but not for SL(2) and this is because the sizes of local $L$-packets for GL(2) are 1, whereas they are of size 1 or 2 for SL(2). Because of this, multipicity 1 is significantly harder to prove for SL(2) than for GL(2).
Now if we move to higher rank, in some sense the closest analogue of GL(2) is GSp(4). You still have discrete series (unlike GL(3) or GL(4)), and we have the exceptional isomorphism PGSp(4) $\simeq$ SO(5) analogous to PGL(2) $\simeq$ SO(3). (Here SO means the split special orthogonal group.) One could also work with other groups with have compact forms (and often does, e.g., unitary groups) but GSp(4) is of interest because it treats the classical theory of Siegel modular forms of degree 2. (Note working on representations of GSp(4) with trivial central character is equivalent to working on PGSp(4).)
Working classically with such Siegel modular forms, you work on Sp(4). Like SL(2), this is slightly more convenient and more general than PSp(4). On the other hand, working representation theoretically, GSp(4) is simpler than Sp(4), similar to the GL(2)/SL(2) situation. E.g., local $L$-packets no longer have size 1, but can have size 1 or 2 on GSp(4), but on Sp(4) they can have size up to 16---see work of Gan and Takeda. So most people who use representation theory find GSp(4) or PGSp(4) easier, and if one is going to work with the more difficult group Sp(4), there is no reason to restrict to PSp(4).
Finally, I don't think that examples are any harder to come up with on GSp(4) rather than PSp(4), due for instance to the connection with SO(5) and theta series. In addition, work on PGSp(4) or GSp(4) may generalize to SO$(2n+1)$.