Let $G$ be a transitive group on $\Omega$. Every orbits of $G$ on its natural action on $\Omega\times\Omega$ is called an orbital of $G$ on $\Omega$. For each orbital $\Delta$ of $G$ on $\Omega$, the orbital digraph $Graph(\Delta)$ is a digraph with vertex set $\Omega$ and edge set $\Delta$. Clearly, $G$ is a subgroup of automorphism group of $Graph(\Delta)$. Is there any paper or book that determine for which groups $G$, there is an orbital $\Delta$ that $G$ is equal to the automorphism group of $Graph(\Delta)$? Clearly, the action of $S_5$ on 2subets of $\{1,\ldots,5\}$ is an example.
The property of $G$ you are looking at is called 2closure, i.e., you ask for a classification of 2closed permutation groups. See the paper by Liebeck, Praeger, Saxl, On the 2closures of finite permutation groups. J. London Math. Soc. (2) 37 (1988), no. 2, 241–252, where this question is investigated for a particular class of primitive permutation groups. More precisely, this is not 100% equivalent to your question (it would be, if you allowed graphs have coloured arcs), as there exist 2closed permutation groups $G$ such that for each orbital the automorphism group of the underlying (di)graph is strictly bigger than $G$. In general such a classification is not known, and the problem is hopelessly hard, I think. 

