For the case $n = p(p-1)(p+1)$ for $p >3$ a prime, a theorem of M. Herzog (to be found in the article "Finite groups with a large cyclic Sylow subgroup" ([MSN](https://mathscinet.ams.org/mathscinet-getitem?mr=338163)) in the conference proceedings "Finite Simple Groups" (Oxford, editors M. Powell and G. Higman, published 1971 by Academic Press ([MSN](https://mathscinet.ams.org/mathscinet-getitem?mr=327886)))) answers your question postively. The Theorem of Herzog uses a 1958 Theorem of Brauer and Reynolds ("[On a problem of E. Artin](https://www.jstor.org/stable/1970164)", Annals of Mathematics, 68 ([MSN](https://mathscinet.ams.org/mathscinet-getitem?mr=100635))) which probably suffices to cover the cases you are interested in.

Note that if $n = p(p-1)(p+1)$ with $p >3$ a prime, and $G$ is a perfect group of order $n$, then $G$ is not $p$-solvable (for if $G$ is $p$-solvable of order $n$, then $G$ either has a factor group of order $p$ or a non-trivial cyclic factor group of order dividing $p-1$).

The Theorem of Herzog/Brauer–Reynolds proves that a finite group $G$ of order less than $p^{3}$ (for a prime $p >3$) which is not $p$-solvable is one of the following groups: $\operatorname{PSL}(2,p)$, $\operatorname{PSL}(2,p-1)$
(for $p >5$ a Fermat prime), $\operatorname{SL}(2,p)$, $\operatorname{PGL}(2,p)$, $\operatorname{PSL}(2,p) \times \mathbb{Z}/2\mathbb{Z}$.

The only perfect group of order $n$ on the list is ${\rm SL}(2.p)$ and the only perfect group of order $\frac{n}{2}$ is ${\rm PSL}(2,p)$, showing that ${\rm SL}(2,p)$ and ${\rm PSL}(2,p)$ ( for $p >3$ a prime) are the unique pefect groups of their orders.