The Weil group appears for several reasons.
Firstly: if $K$ is a non-archimedean local field with residue field $k$,
the local reciprocity law induces an embedding
$K^{\times} \hookrightarrow G_K^{ab}.$ The image consists of all
elements in $G_k$ whose image is an integral power of Frobenius.
This is the abelianized Weil group; it just appears naturally.
Secondly: suppose that $K$ is a global field of positive characteristic,
i.e. the function field of a curve over a finite field $k$. Then the
global reciprocity map identifies the idele class group of $K$ with
a subgroup of $G_K^{ab}$ consisting of elements which act on $k$
by integral powers of Frobenius. So again, it is the abelianized Weil group
that appears.
Thirdly: suppose that $E$ is an elliptic curve over a quadratic imaginary
field $K$
with complex multipliction by $\mathcal O$, the ring of integers in $K$. (Thus I am implicity fixing $K$ to
have class number one, but this is not so important for what I am going to say next.)
If $\ell$ is a prime, then the $\ell$-adic Tate module is then free of rank one over
$\mathcal O_{\ell}$ (the $\ell$-adic completion of $\mathcal O$), and the $G_K$-action
on this Tate module induces a character $\psi_{\ell}:G_K^{ab} \rightarrow \mathcal O_{\ell}^{\times}$.
There is a sense in which the various $\psi_{\ell}$ are indepenent of $\ell$,
but what is that sense?
Well, suppose that $\wp$ is a prime of $K$, not dividing $\ell$ and at which
$E$ has good reduction. Then the value of $\psi_{\ell}$ on $Frob_{\wp}$ is indepenent
of $\ell$, in the sense that its value is an element of $\mathcal O$, and this value
is independent of $\ell$.
More generally, provided that $\wp$ is prime to $\ell$,
the restriction of $\psi_{\ell}$ to the local Weil group at $\wp$ is independent of $\ell$
(in the sense that the value at a lift of Frobenius will be an algebraic integer that
is independent of $\ell$, and its restriction to inertia at $\wp$ will be a finite image
representation, hence defined over algebraic integers, which again is then independent
of $\ell$).
Note that independence of $\ell$ doesn't make sense for $\psi_{\ell}$
on the full local Galois group at $\wp$, since on this whole group it will
certainly take values that are not algebraic, but rather just some $\ell$-adic
integers, which can't be compared with one another as $\ell$ changes.
Now there is also a sense in which the $\psi_\ell$, as global Galois characters, are independent of $\ell$. Indeed, we can glue together the
various local Weil group representations to get a representation $\psi$ of the global Weil
group $W_K$. Since it is abelian, this will just be an idele class character $\psi$,
or what is also called a Hecke character or Grossencharacter. It will take values
in complex numbers. (At the finite places it even takes algebraic number values, but
when we organize things properly at the infinite places, we are forced to think of it
as complex valued.)
Note that $\psi$ won't factor through the connected component group, i.e. it won't be
a character of $G_K^{ab}$. It is not a Galois character, but a Weil group character.
It stores in one object the information contained in a whole collection of
$\ell$-adic Galois characters, and gives a precise sense to the idea that these various $\ell$-adic characters are independent of $\ell$.
This is an important general role of Weil groups.
Fourthly: The Hecke character $\psi$ above will be an algebraic Hecke character, i.e.
at the infinite places, it will involve raising to integral powers. But we can also
raise real numbers to an arbitrary complex power $s$, and so there are Hecke characters
that do not come from the preceding construction (or ones like it); in other words,
there are non-algebraic, or non-motivic, Hecke characters. But they are abelian characters
of the global Weil group, and they have a meaning; the variable $s$ to which we can raise
real numbers is the same variable $s$ as appears in $\zeta$- or $L$-functions.
In summary: Because Weil groups are "less completed", or "less profinite", than Galois
groups, they play an important role in describing how a system of $\ell$-adic representations can be independent of $\ell$. Also, they allow one to describe
phenomena which are automorphic, but not motivic (i.e. which correspond to non-integral
values of the $L$-function variable $s$). (They don't describe all automorphic phenomena,
though --- one would need the entire Langlands group for that.)
