Does the linear automorphism group determine the vector space? I was recently thinking about what it means to put structure on a set.  It seems to me that, in my area (representation theory), the two main ways of imposing structure on a set $X$ are:


*

*distinguishing certain permutations of $X$ as structure preserving;


and


*

*distinguishing certain test functions (here I think of $X \to \mathbb C$, but that's just because it fits my example) are structure respecting.


For example, given a manifold, we can look at the diffeomorphisms among its permutations, or the smooth functions among its test functions; and, given a vector space, we can look at the linear automorphisms among its permutations, or the functionals among its test functions.  Under reasonable hypotheses, the test-functions perspective (more 'analysis'-flavoured, maybe) determines the structure; but does the group perspective (more 'geometry' / Erlangen-flavoured)?
A colleague pointed out that, for manifolds, the diffeomorphism group does determine the manifold, in a very strong sense:  http://www.ams.org/mathscinet-getitem?mr=693972.  Given that success, I was moved to ask:  does the linear automorphism group determine the vector space?
Here's one way of making that informal question precise.  Suppose that $V_1$ and $V_2$ are vector spaces (over $\mathbb C$, say) with the same underlying set $X$, and that the set of permutations of $X$ that are linear automorphisms for $V_1$ is the same as the analogous set for $V_2$.  Then are $V_1$ and $V_2$ isomorphic?
Another colleague, and The Masked Avenger, both thought of the axiom of choice when I asked this question; but I'm not sure I see it.  It's just a curiosity, so I have no particular investment in whether answers assume, negate, or avoid choice.
EDIT:  Since I think it may look like I am making some implicit assumptions, I clarify that I do not mean to assume that the vector spaces are finite dimensional, or that the putative isomorphism from $V_1$ to $V_2$ must be the identity as a set map of $X$.  Thus, for example, Theo Johnson-Freyd's example (https://mathoverflow.net/a/186494/2383) of letting $X$ be $\mathbb C$, and equipping it with both its usual and its 'conjugate' $\mathbb C$-vector space structure, is perfectly OK.
 A: Vladimir Dotsenko is right:

$GL(V)\cong GL(W)$ iff $V\cong W$. 

This is true for any complex (and not only complex) vector spaces. 
If at least one of the dimensions is finite, then this follows from Dotsenko's argument. If $\dim V$ and $\dim W$ are infinite, this is valid for vector spaces over arbitrary division rings by
Tolstykh's theorem [Theorem 8.3] (the paper contains a complete proof of the 
theorem and a remark that it "easily follows from general isomorphism theorems proved by O’Meara in [15, Theorem 5.10, Theorem 6.7]").
[15] O. O’Meara, A general isomorphism theory for linear groups, J. Algebra, 44 (1977) 
93-142.   
A: A simple argument in the finite dimensional case: the commutator subgroup of $GL_n(\mathbb{C})$ is $SL_n(\mathbb{C})$, and the size of the center of $SL_n(\mathbb{C})$ is $n$, as scalar matrices with determinant 1 correspond to roots of unity. (This prompts a question as to whether the general argument can be arranged in a way that uses just the group structure of $GL(V)$, not the action of $GL(V)$ on $V$).
A: No, of course not, at least not the way the question is posed.  For example, consider a one-dimensional complex vector space $V \cong \mathbb C^1$, and the complex conjugate vector space $\bar V$, which as an abelian group is the same as $V$, but has the complex conjugate action.  I.e. for an element $v\in V$, the action of $\lambda \in \mathbb C$ on $v$ when thought of as an element of $\bar v$ is the action of $\bar \lambda$ on $v$ when thought of as an element of $V$.  These two vector space structures are not literally the same structure, but their automorphism groups are literally the same subgroup of the group of all permutations of the underlying set of $V$.
You can come up with a similar example for any vector space over any field that admits automorphisms.  To rule out this problem, we can study the similar question over a field like $\mathbb Q$ or $\mathbb R$ that does not admit any automorphisms, or we can ask whether perhaps this is the only failure to detect the vector space structure (i.e. modify the goal, so that the reconstruction is considered a success if the two vector space structures differ only by some pullback of scalar multiplication along a field automorphism).
Over $\mathbb Q$, the action by $\mathbb Q$ is no data — vector spaces are determined by their structure as additive groups — and Clement de Seguins Pazzis has shown in a previous answer that the additive structure can be determined from the group of automorphisms.  At least, he shows this when the dimension is greater than $1$.
Actually, Clement's argument fails in dimension $1$.  Consider $V = \mathbb Q^1$.  Then $\mathrm{GL}(1,\mathbb Q) = \mathbb Q^\times$ acts simply transitively on $V \smallsetminus \{0\}$.  The fundamental theorem of arithmetic then identifies $\mathbb Q^\times$ as the free abelian group with the set of primes as a basis.  In particular, $\mathbb Q^\times$ itself has many complicated automorphisms — you could completely rearrange the primes, for example.  Via any such automorphism, you can define an addition structure on $\mathbb Q^\times \sqcup \{0\}$ whose automorphism group is the group you started with.  Here is an explicit example: write a non-zero rational number as $2^a 3^b \frac p q$ with $a,b\in \mathbb N$ and neither $p$ nor $q$ divisible by either $2$ or $3$.  Define $\tau(2^a 3^b \frac p q) = 2^b 3^a \frac p q$, and define $+_\tau$ by $x +_\tau y = \tau(\tau(x)+\tau(y))$.  Since $\tau$ is an intertwiner for the $\mathbb Q^\times$-action on $\mathbb Q$, this will define a nontrivial addition on $\mathbb Q$ with the same automorphism group.
A: A. Prasad shows that one has access to the dimension $n$ of $V$. 
Assuming that $n$ is greater than $1$, the addition on $V$ can be recovered from the knowledge of which permutations of $V$ are automorphisms. 
First of all, we can recover, for a given subset $F$ of $V$, the span of $F$:
it is the set of all vectors $x$ that are fixed by all $g \in GL(V)$
that already fix all the vectors in $F$. 
In particular, we can use this to determine which subsets are linear subspace; in particular, we know what the zero vector is, which subsets are 
are $1$-dimensional linear subspaces and which subsets are linear hyperplanes. 
Next, we can determine which subsets are affine subspaces and what their translation vector space is:
let $H$ be a linear hyperplane of $V$ : we consider the subgroup of $GL(V)$
consisting of the identity and of the automorphisms that fix all the vectors of $H$ 
but stabilizes no $1$-dimensional linear subspaces that is not included in $H$ (in short: it is the group of transvections with respect to $H$). 
Then, the affine hyperplanes that are parallel to $H$ are the orbits of $V \setminus H$
under the action of this subgroup. By writing every affine subspace as an intersection of affine hyperplanes, we understand which subsets are affine subspaces and for each such affine subspace we know what its direction is. 
From there, we recover addition. First of all, 
for each vector x, the opposite of $x$ should be $f(x)$
if $f$ is the sole element of the center of $GL(V)$ that satisfies $f^2=id$ and is different from $id_V$, and if no such $f$ exists we have $-x=x$. 
On the other hand, given non-collinear vectors $x$ and $y$
(non-collinearity is tested by saying that $span(x) \neq span(y)$, $x \neq 0$ and $y \neq 0$), we see that $x+y$ is the sole common point of the affine line that goes through $x$ and is parallel to $span(y)$ and the one that goes through $y$ and is parallel to $span(x)$. Since the addition is associative, we can use this to understand how the addition works on non-zero collinear vectors: given such vectors $x$ and $y$, we choose $z$ that is non-collinear to $x$, and we compute
$x+y=((x+z)+y)+(-z)$. 
A: The dimension of $V$ is the least non-negative integer $n$ such that there exist $v_1,\dotsc, v_n$ in $V$ such that there exists a unique $g\in G:=GL(V)$ that fixes each of $v_1,\dotsc,v_n$. So the isomorphism class of $V$ is determined by the group action of $G$ on $V$.
A: So I asked a colleague and with his permission I'll post an argument using choice and GCH. 
We just need to find a bijection between the bases (which exists by choice). 
If $V$ and $W$ are finite dimensional, the size of $GL(V)$ already distinguishes them.
Edit to clarify the finite dimensional situation because of all the comments and this even got downvoted:
Let $S$ be a set and $V$ and $W$ be two linear vector space structures on $S$. Let us suppose that the group of linear automorphisms of these structures is the same. We are supposed to prove that $V$ is linearly isomorphic to $W$. Lets suppose that they are both finite dimensional vector spaces, say of dimension $n$ and $m$ respectively. If $n=m$ there is nothing to prove. In the contrary case lets proceed by contradiction. The group of automorphisms of $V$ is isomorphic to $GL(n)$. The group of automorphisms of $W$ is isomorphic to $GL(m)$, as $GL(n)\neq GL(m)$ we are done.
Now to the actual non-trivial part of the problem:
If $V$ and $W$ have infinite bases, then we first note that the set of finite parts of an infinite set $X$ has the same cardinality $|X|$ of $X$. The cardinality $|k|$ of our field $k$ (say we have $k=\mathbb{C}$) raised to a finite power is equal to $|k|$. So if $X$ is a basis of the space $V$ we have that $|V| = max(|k|, |X|)$. 
Suppose now that the cardinality of the basis $X$ for $V$ is bigger than $|k|$. We see that $|V|=|W|$ implies that $|X|=|Y|$ where $Y$ is a basis of $W$ and we are done (No group of linear transformations was needed here!)
So now we need to check the situation when $|X|=k$ or $X$ is numerable (by CH). That is we want to show that it is absurd to have $X$ numerable and $|Y|=|k|$. Here we will look at the sizes of the groups of automorphisms. As we have that these groups have the sizes of permutations of bases, or $2^{|X|}$ and $2^{|Y|}$. respectively, we have that (in general by GCH) $|X| < |Y|$ implies $2^{|X|} < 2^{|Y|}$ and we are done. 
