When studying knot theory I often encounter $SL(2, \mathbb{C})$representation of knots (of the knot group) or the $SL(2, \mathbb{C})$ character variety of a knot group. But I just don't seem to understand what this is all about and when the special linear group comes into play. Can anyone recommend me literature that covers the basics to this topic and where to start? Perhaps a nice gentle introduction preferably with examples?

2$\begingroup$ Are you familiar with the theory of hyperbolic structures on knot complements? Thurston's notes is a good place to start, especially Ch 4 library.msri.org/books/gt3m $\endgroup$– NealFeb 12, 2019 at 17:02

1$\begingroup$ Thurston's work is a very natural generalization of perhaps the "first" way of studying knots: compute the fundamental group of the knot exterior. But then how do you know your knot is nontrivial? Find a representation to some nonabelian matrix group. $\endgroup$– Ryan BudneyFeb 14, 2019 at 16:18
3 Answers
$(P)SL(2, \mathbb{C})$ is the isometry group of $\mathbb{H}^3,$ so $SL(2, \mathbb{C})$ representations are the natural generalization of hyperbolic structures on knot complements.There is a vast literature on the subject, but you might want to look at some of the foundational work:
Morgan, John W.; Shalen, Peter B., Valuations, trees, and degenerations of hyperbolic structures. I, Ann. Math. (2) 120, 401476 (1984). ZBL0583.57005.
The fundamental group of the complement of a knot in $S^3$, called a knot group, is a knot invariant (equivalent knots have the same knot group, but not conversely). To understand knot groups, understanding their representations is important (and the moduli spaces of their representations; that is, their character varieties). Representations into $\mathrm{SL}(2,\mathbb{C})$ have a special place among such representations since many (but perhaps not most) of the 3manifolds that corresponds to knot complements are hyperbolic (and $\mathrm{PSL}(2,\mathbb{C})$ is the isometry group of real hyperbolic 3space).
But such representations, and the correspondence to hyperbolic geometry, is really the tip of the iceberg.
Adam Sikora has many papers relating knot theory to $G$character varieties (not only to $\mathrm{SL}(2,\mathbb{C})$character varieties; example 1 and example 2). In particular, as a starting point, I recommend:
On Skein Algebras And SL(2,C)Character Varieties by Józef Przytycki and Adam Sikora.
This paper ties together $\mathrm{SL}(2,\mathbb{C})$character varieties to Kauffman bracket skein modules, the latter being an important 3manifold invariant in knot theory (for example, the Jones polynomial can be defined in terms the Kauffman bracket).
Another interesting connection is the Apolynomial, also defined in terms of $\mathrm{SL}(2,\mathbb{C})$character varieties. See Representation Theory and the Apolynomial of a Knot, by Cooper and Long, for a nice introduction. Also, Culler has made available a census of Apolynomials here.
I would argue that $\mathrm{SL}(2,\mathbb{C})$character varieties of knot groups are prevalent in the literature for many reasons (including the relation to hyperbolic geometry), but also perhaps because they are tractable examples of moduli spaces of representations (see my recent exposition here, implemented in SnapPy here). I expect that $G$character varieties of knot groups (these moduli space are themselves knot invariants) are at least as important (for higher rank $G$), but are less studied since they are much more difficult to work with.
This is a good introduction:
Shalen, Representations of 3manifold groups. Handbook of geometric topology, 955–1044, NorthHolland, Amsterdam, 2002.