If $G$ is a non-abelian group, then it has a non-trivial inner automorphism (conjugation by any non-central element). If $G$ is abelian of exponent bigger than 2, then the inversion map is an automorphism. If $G$ is of exponent 2, then it is a vector space over the 2-element field $F$, of dimension at least 2 (assuming that $|G|>2$). Choose a basis for $G$; then the map interchanging the first two basis vectors and fixing the rest extends to an automorphism.
This is fine in ZFC, but if we don't assume the Axiom of Choice there can be vector spaces without bases. A weaker condition suffices: if the dual space of $G$ is non-zero, then $G$ has a non-trivial automorphism. (Choose $f\in G^*$, $f\ne0$, and $a\in\ker(f)$; then the transvection $x\mapsto x+f(x)a$ is a non-trivial automorphism. Can there be a non-zero vector space with zero dual space?
To answer the original question, we need to know: can there exist a vector space (containing two linearly independent vectors) all of whose automorphisms are scalars? And can such a vector space exist over the 2-element field?
I suspect that these things can exist in suitable models of ZF. Does anyone know a reference?