This is a fairly long-standing question in certain quarters, though I would need to check who was the first to ask it (if such a person is well-defined). Isaacs and Karagueuzian answered a somewhat related question in the negative (around 2002), disproving a conjecture of Kirillov. They proved that a Sylow $2$-subgroup of ${\rm GL}(13,2)$ is not a real group. (Recall that the definition of a real group is analagous to the definition of a rational group given in the question. A finite group is real if and only if all its complex irreducible characters are real-valued, which is equivalent to all its elements being conjugate to their inverses). However, I should point out that in my original post, I had mis-remembered the content of the Isaacs-Karagueuzian result. Contrary to my earlier statement, the group ${\rm GL}(13,2)$ is not itself a real group. For example, it contains an element of order $127$ which is not conjugate to its inverse. As far as I am aware, the given question about rational groups is open. One of the difficulties with the question is that rational groups are relatively rare (a loose statement, I know, but justifiable).

`$S_4$`

and its`$2$`

-Sylow subgroups. – Jim Humphreys Jun 8 '11 at 13:48