The answer is "yes", see below.
Dieudonné in his book "La géométrie des groupes classiques" considers the abstract group $SL_n(K)$ for a field $K$, not necessarily commutative, and writes $PSL_n(K)$ for $SL_n(K)$ modulo the center. In Ch. IV, Section 9, he considers the question whether $PSL_n(K)$ can be isomorphic to $PSL_m(K')$ for $n\ge 2,\ m\ge 2$. He writes that they can be isomorphic only for $n=m$, except for $PSL_2(\mathbb{F}_7)$ and $PSL_3(\mathbb{F}_2)$. If $n=m>2$, then the isomorphism is possible only if $K$ and $K'$ are isomorphic or anti-isomorphic. The same is true for $m=n=2$ if both $K$ and $K'$ are commutative, except for the case $K=\mathbb{F}_4$, $K'=\mathbb{F}_5$. Dieudonné gives ideas of proof and references to Schreier and van der Waerden (1928), to his paper "On the automorphisms of classical groups" in Mem. AMS No. 2 (1951) and to the paper of Hua L.-K. and Wan in J. Chinese Math. Soc. 2 (1953), 1-32.
This answers affirmatively the question for $SL_n$, because if $SL_n(K)\cong SL_n(K')$, then $PSL_n(K)\cong PSL_n(K')$. In the case $n=2$, $K=\mathbb{F}_4$, $K'=\mathbb{F}_5$, the orders $|SL_2(\mathbb{F}_4)|=60$ and $|SL_2(\mathbb{F}_5)|=120$ are different, and therefore thethese groups are not isomorphic.
This also answers affirmatively the question for $GL_n$, because $SL_n(K)$ is the commutator subgroup of $GL_n(K)$, except for $GL_2(\mathbb{F_2})$, see Dieudonné, Ch. II, Section 1. In the case $n=2$, $K=\mathbb{F}_2$, the groupwe have $GL_2(\mathbb{F}_2)$ is of order 6$|GL_2(\mathbb{F}_2)|=6$ , and one sees easily that its order iswhich is less than the order $q(q-1)(q^2-1)$ of $GL_2(\mathbb{F}_q)$ for$|GL_2(\mathbb{F}_q)|=q(q-1)(q^2-1)$ for any $q=p^r>2$, hence $GL_2(\mathbb{F}_2)\not\cong GL_2(\mathbb{F}_q)$ for $q>2$.