Is it possible to classify finite simple groups whose every maximal subgroups are not of prime order? Is it possible to answer to this question in the class of finite groups?
Prop: If $G$ is a finite simple group, then a maximal subgroup of $G$ is trivial or has composite order
Proof: A maximal subgroup of $G$ being trivial clearly corresponds to $G$ being cyclic of prime order. Assume, then, that $G$ is non-abelian.
If $G$ has a maximal subgroup $C$ of prime order, then the action of $G$ on cosets of $C$ is Frobenius. Thus $G$ is a Frobenius group, $C$ is a Frobenius complement and $G$ contains a Frobenius kernel, i.e. $G$ is not simple. QED
So this answers your first question. As for your more general question about finite groups. Well, again, if a group has a maximal subgroup of prime order, then it is Frobenius, so you should consult the literature on Frobenius groups. For this I particularly recommend Isaac's "Finite Group Theory" and Passman's "Permutation groups".
Examples of groups with a maximal subgroup of prime order include dihedral groups of order $2m$ ($m$ odd) or, more generally $C_n \rtimes C_p$ where $p$ is a prime and $C_p$ acts semi-regularly on $C_n$.