I have a really hard time "feeling" what it means for a group to fail to be linear. Vaguely, I'd like to know how one should think about such groups. More precisely:
What are some interesting examples of groups that aren't linear?
Are there general constructions that one can use to cook up a group or a family of groups that isn't linear?
Are there general techniques that one can use to show that a given group isn't linear?
More loosely, what is it about these groups that makes them interesting?
Consider the class of finitely generated linear groups. Such groups $G$ satisfy certain well-known restrictions, for instance:
Every such $G$ is residually finite (Malcev, 1940). Thus, most Baumslag-Solitar groups, e.g. $$ \langle a, b| a b^2 a^{-1} =b^3\rangle $$ are not linear. This is the simplest example of a nonlinear f.g. group I know.
$G$ is virtually torsion-free (Selberg, 1960). In particular, if $G$ is torsion then it is finite (which was known to Burnside). Note that there are infinite torsion residually finite groups (first examples are due to Golod and Shafarevich); such groups have to be nonlinear.
$G$ satisfies Tits' alternative (Tits, 1972): Either $G$ contains a free nonabelian subgroup or contains a solvable subgroup of finite index. (Thus, for instance, Thompson group is not linear.)
Tarski mosters will violate all of the above restrictions.
There are more subtle restrictions, for instance, $Aut(F_n), n\ge 3$ is not linear (Formanek and Procesi, 1992).
Consider reading Wehrfritz' book "Infinite linear groups" or this survey to get a better idea of what linearity means for f.g. groups, specially, Lubotzky's criterion of linearity.
Concerning your question of why nonlinear groups are interesting: Many of them occur naturally (like $Aut(F_n)$), the rest push the boundaries of our understanding of the class of f.g. groups. For many "natural" groups, linearity is unknown, e.g., the mapping class group $Mod_g$, $g\ge 3$.
The following is an elaboration of the last paragraph of Misha's answer.
For me, the thing that makes non-linear (discrete) groups interesting is that we are not very good at constructing them! Linearity remains unknown for many natural examples of groups, such as mapping class groups, and if true would dramatically simplify some hard theorems. (For instance, Daniel Groves has a very long and difficult proof that mapping class groups are 'equationally Noetherian'; if they are linear then it is an easy consequence of Hilbert's Basis Theorem.)
Similarly, we only know one way of constructing non-linear word-hyperbolic groups (noticed by Misha, in fact): take a uniform lattice $\Gamma$ in Sp(n,1), which is both word-hyperbolic and satisfies Margulis super-rigidity, and kill a 'random' element; the resulting quotient $Q$ is an infinite quotient of $\Gamma$ with infinite kernel, so by Margulis super-rigidity cannot be linear (at least in zero characteristic; I'm not sure about characteristic $p$).
The fact that we know no other methods of constructing non-linear groups is related to the fact that we do not know how to construct a non-residually finite word-hyperbolic group.
Surprising recent developments partially explain this failure by showing that linear (and hence residually finite) groups are much more common than we thought. I think most experts would have guessed that a 'random' finitely presented group (which is known to be word-hyperbolic) would not be linear; in fact, it follows from recent work of Agol and older work of Wise that some parts of the 'spectrum' of random groups are in fact linear. These are exciting times!
The universal covering $G^*$ of the group $SL_2({\mathbb R})$ is not linear. The reason is that any linear representation of $G^*$ is given by a Lie algebra representation, which descends to a representation of $SL_2({\mathbb R})$. The essential reason is that $SL_2({\mathbb C})$ is simply connected.
More generally, if $G$ is a simple algebraic group defined over ${\mathbb R}$ such that $G({\mathbb C})$ is simply connected, and if $G({\mathbb R})$ is not simply connected, then the universal cover of $G({\mathbb R})$ is not linear.
If you have a finitely generated linear group whose growth is sub-exponential, then by Tits alternative (mentioned above) the group has to be virtually solvable and by a theorem of Milnor the group has polynomial growth. Hence any finitely generated group whose growth is sub-exponential and not polynomial (so called intermediate growth) is not linear. There are many examples of groups with intermediate growth.
It's not clear whether you are working in the setting of Lie theory, or abstract group theory, or something else. This answer addresses the Lie theory aspect of the matter.
Let's focus on Lie groups whose Lie algebra is semisimple (as solvable radicals mess up things in too many ways, as usual). By serious theorems, the functor $\mathbf{G} \rightsquigarrow \mathbf{G}(\mathbf{C})$ from connected semisimple $\mathbf{C}$-groups to connected complex Lie groups with semisimple Lie algebra is an equivalence of categories. So all connected complex Lie groups with semisimple Lie algebra admit a unique and functorial "linear algebraic" structure.
Let's say that a connected Lie group $G$ with semisimple Lie algebra is linear if $G = \mathbf{G}(\mathbf{R})^0$ for a connected semisimple $\mathbf{R}$-group $\mathbf{G}$. From the viewpoint of "semisimple" Lie theory, the failure of this condition is a bit tricky to think about because non-isomorphic connected semisimple $\mathbf{R}$-groups can yield isomorphic connected Lie groups of $\mathbf{R}$-points, the most famous being the degree-$n$ isogeny ${\rm{SL}}_n \rightarrow {\rm{PGL}}_n$ over $\mathbf{R}$ with an odd $n > 1$ (this becomes an isomorphism on $\mathbf{R}$-points). Nonetheless, we can characterize it in terms of the complex-analytic theory as follows.
Consider a connected Lie group $G$ over $\mathbf{R}$ whose Lie algebra $\mathfrak{g}$ is semisimple. Dropping any semisimplicity hypotheses on Lie algebras for a moment, there is a general notion of complexification of $G$, namely a homomorphism $r_G:G \rightarrow G_{\mathbf{C}}$ to a complex Lie group $G_{\mathbf{C}}$ that is initial among all homomorphisms $\rho:G \rightarrow H$ to a complex Lie group (i.e., there is a unique holomorphic homomorphism $f:G_{\mathbf{C}} \rightarrow H$ such that $f \circ r_G = \rho$). This is constructed in complete generality in Bourbaki LIE Chapter III, for example. In general ${\rm{Lie}}(G_{\mathbf{C}})$ is a quotient of $\mathfrak{g}_{\mathbf{C}}$, so when $\mathfrak{g}$ is semisimple this quotient is semisimple and hence $G_{\mathbf{C}}$ is (canonically) linear over $\mathbf{C}$.
Obviously if $G = \mathbf{G}(\mathbf{R})^0$ for a connected semisimple $\mathbf{R}$-group $\mathbf{G}$ then the resulting closed embedding $G \rightarrow \mathbf{G}(\mathbf{C})$ factors uniquely through a holomorphic map $G_{\mathbf{C}} \rightarrow \mathbf{G}(\mathbf{C})$ via composition with $r_G$, so $\ker r_G = 1$.
Remarkably, the converse holds: if $\ker r_G = 1$ then $G$ is the identity component of the group $\mathbf{G}(\mathbf{R})$ of $\mathbf{R}$-points of a connected semisimple $\mathbf{R}$-group $\mathbf{G}$ (and $r_G$ is actually a closed embedding). Indeed, the canonical "algebraization" of $G_{\mathbf{C}}$ has Weil restriction $G'$ over $\mathbf{R}$ that is a connected semisimple $\mathbf{R}$-group such that $r_G$ is identified with an injective map $G \rightarrow G'(\mathbf{R})$ between connected Lie groups. In particular, $\mathfrak{g}$ is identified with a semisimple Lie subalgebra of ${\rm{Lie}}(G')$, so by the algebraic theory over $\mathbf{R}$ (as over any field of characteristic 0) it has the form ${\rm{Lie}}(\mathbf{G})$ for a unique connected semisimple closed $\mathbf{R}$-subgroup $\mathbf{G} \subset G'$. Thus, $r_G$ factors through $\mathbf{G}(\mathbf{R})^0$. The resulting injective map $G \rightarrow \mathbf{G}(\mathbf{R})^0$ between connected Lie groups is an isomorphism on Lie algebras and thus is surjective, so it is an isomorphism of Lie groups.
The upshot is that a connected Lie group $G$ with semisimple Lie algebra is linear if and only if $\ker r_G = 1$, in which case $r_G$ is a closed embedding. You may therefore think of the non-triviality of $\ker r_G$ (i.e., the absence of "enough" homomorphisms to complex Lie groups to separate points) as the exact obstruction to $G$ being linear in the sense defined above.
Let me give you two examples from the theory of pro-$p$ groups:
The Nottingham group contains every finite $p$-group. Thus, it is not linear over any field.
Zubkov proved that a non-abelian free pro-$p$ groups cannot be embedded into $SL_2(R)$, where $R$ is any pro-$p$ ring. This implies that any pro-$p$ subgroup of $SL_2(R)$ satisfies a pro-$p$ identity. Lubotzky and Shalev conjectured that this is true for $SL_n(R)$ for any $n$ and any pro-$p$ ring $R$. Micheal Larsen and I proved that a non-abelian free pro-$p$ group is not linear over local fields. Our proof was a fairly easy application of Richard Pink's work which enabled us to reduce the question to showing that an open subgroup of semisimple group over local fields is not free. I think we then used some result of Lubotzky, but I am sure there are many arguments to deal with that case. Let me emphasize that our result does not imply the existence of a pro-$p$ identity (for that you need to deal with all pro-$p$ rings). However, Zelmanov claims in an unpublished work that pro-$p$ identity exists if we fix $n$ and let $p$ be big enough. Also, notice that this does not imply non-linearity in general.
If $A$ is any abelian variety, i.e. a proper algebraic variety that is also an algebraic group, then it is not a linear algebraic group, because any map $A \to \operatorname{GL}_n$ is a map from $A$ to an affine variety and is thus constant (so not faithful). This sort of does not answer your question, as $A$ is not a single group, though it does mean that the group in sets $\newcommand\C{\mathbb{C}}A(\C)$ (or the appropriate algebraically closed field of positive characteristic) is not linear. However, interestingly, it is not the case that this gives a whole family of non-linear groups, since for example if $A$ is defined over $\newcommand\Z{\mathbb{Z}}\Z$, then $\newcommand\F{\mathbb{F}}A(\F_q)$ is a finite abelian group for any prime power $q$ and, easily, is linear, though their faithful representations won't combine into a faithful representation of $A(\bar{\F}_q)$.
