If a group $G$ has a maximal subgroup $C$ of prime order, then the action of $G$ on cosets of $C$ is Frobenius. Thus $G$ is a Frobenius group, $C$ is a Frobenius complement and $G$ contains a Frobenius kernel, i.e. $G$ is not simple, and your first question is answered.

As for your more general question about finite groups: There is loads of stuff in the literature about Frobenius groups, so you should do some reading! I particularly recommend Isaac's "Finite Group Theory" and Passman's "Permutation groups".

Examples of groups with a maximal subgroup of prime order include dihedral groups of order $2m$ ($m$ odd) or, more generally $C_n \rtimes C_p$ where $p$ is a prime and $C_p$ acts semi-regularly on $C_n$.

Prop: If $G$ is a finite simple group, then a maximal subgroup of $G$ is trivial or has composite order

Proof: A maximal subgroup of $G$ being trivial clearly corresponds to $G$ being cyclic of prime order. Assume, then, that $G$ is non-abelian.

If $G$ has a maximal subgroup $C$ of prime order, then the action of $G$ on cosets of $C$ is Frobenius. Thus $G$ is a Frobenius group, $C$ is a Frobenius complement and $G$ contains a Frobenius kernel, i.e. $G$ is not simple. QED

So this answers your first question. As for your more general question about finite groups. Well, again, if a group has a maximal subgroup of prime order, then it is Frobenius, so you should consult the literature on Frobenius groups. For this I particularly recommend Isaac's "Finite Group Theory" and Passman's "Permutation groups".

Examples of groups with a maximal subgroup of prime order include dihedral groups of order $2m$ ($m$ odd) or, more generally $C_n \rtimes C_p$ where $p$ is a prime and $C_p$ acts semi-regularly on $C_n$.