Lets fix an orbit $X = G/H$. It suffices to determine the asymptotic growth of the trace of multiplication by $X^n$, since the maximal positive eigenvalue(s) dominates the sum $\sum_i \lambda_i^n$.
This trace is the sum $$\sum_{K \subset G} \langle G/K, X^n \times G/K \rangle,$$ where the sum is over conjugacy classes of subgroups, and the bracket counts the number of disjoint orbits of type $G/K$.
Notice that the term $\langle G, X^n \times G \rangle$ equals $|X|^n$ (one free orbit for each $x^n \times e$). Conversely, $\langle G/K, X^n \times G/K \rangle$ must have size $\leq |X|^n |G/K|/|G/K| = |X|^n$.
Thus the exponent of the order growth is $|X|$, and so $|X|$ is the maximal eigenvalue. The class $[G]$ is an eigenvector corresponding to this eigenvalue.
Edit: As Darij points out, in this case Perron--Frobenius only guarantees that $$ max_{\lambda \in {\rm eig}(A)} |\lambda|$$ can be achieved by some positive real $\lambda$. There may be other complex eigenvalues of the same absolute value. In this case more argument is required to ensure that there is not cancellation.
Edit2: Here is a strategy to resolve the issue. Let $z_1, \dots, z_k$ be the eigenvalues achieving the maximum. Assume we can show that there exists an eplison such that $$S = \{n \in \mathbb N ~|~ Re(z_i^n) \geq -\lambda + \epsilon ~ \forall i \}$$ is infinite. Then there would be an infinite subset $S \subset \mathbb N$ and a $C \in \mathbb R_{> 1}$ such that $C \lambda^s \geq Tr(A^s) \geq 1/C \lambda^s$ for all $s \in S$.
In our case, we have $$|X|^n \leq {\rm Tr}(A^n) \leq \#\{\text{ subgroups }\}|X|^n$$ for all $n$. Restricting to the infinite subset $S$ we would see that $\lambda = |X|$.