Representations of the Lorentz group The first few lines of this post is based on this lecture notes, but similar expositions can be found in other physics books such as Peskin & Schroeder's book.
On chapter 8 of the linked notes, the author is interested on finding all representations of the Lorentz group which make the physics of the system invariant.
Let me sketch the main points of his exposition. A small Lorentz transformation is:
\begin{eqnarray}
\Lambda_{\nu}^{\mu} = \delta_{\nu}^{\mu} + \omega_{\nu}^{\mu} \tag{1}\label{1}
\end{eqnarray}
where the parameters of the transformation satisfy $\omega_{\nu}^{\mu} = -\omega_{\mu}^{\nu}$. Any $n\times n$ matrix representation of such transformation on fields $M_{ab}(\Lambda_{\nu}^{\mu}) = M_{ab}(\delta_{\mu}^{\nu}+\omega_{\nu}^{\mu})$ is expanded:
\begin{eqnarray}
M_{ab}(\Lambda_{\nu}^{\mu}) = M_{ab}(\delta_{\nu}^{\mu}) + \sum_{\mu< \nu}\omega_{\mu\nu}\frac{\partial M_{ab}(\Lambda)}{\omega_{\mu\nu}}  + O(\omega^{2}) \tag{2}\label{2}
\end{eqnarray}
Then, define the generators of the representation:
\begin{eqnarray}
(J^{\mu\nu}_{M})_{ab} := i\frac{\partial M_{ab}(\Lambda)}{\omega_{\mu\nu}}  \tag{3}\label{3}
\end{eqnarray}
and exponentials of those $J$ are supposed to generate all Lorentz transformations.
Now, the above scenario seems very non-rigorous. To begin with, a representation of a group $G$ is a homomorphism $\rho : G \to GL(V)$ where $V$ is a vector space and $GL(V)$ the group of all linear transformations from $V$ to itself (the product being the composite of functions). Thus, I think that by "All representations of the Lorentz group" physicists mean something like every element of $GL(V)$ for which the theory is invariant.
In any case, I'd like to know if there exists a mathematical precise formulation of the above. I have no idea what even to look for on the internet, so any help is appreciated. Could someone explain the details of it or provide good references, if possible? In special, I'd like to understand what are these $J$ given by (\ref{3}), what they generate and so on. More importantly, for the scalar representation, $J$ should be:
\begin{eqnarray}
J^{\mu\nu} = -i(x^{\mu}\partial^{\nu}-x^{\nu}\partial^{\mu}) \tag{4}\label{4}
\end{eqnarray}
and this seems to be of special interest in the theory, understanding (\ref{4}) is very important to me.
 A: Since your question is now asking for references, here are a few standard ones.
For those who wish to study Lie groups and Lie algebras for the purposes of representation theory (one already mentioned by gmvh):


*

*Fulton, William; Harris, Joe, Representation theory. A first course, Graduate Texts in Mathematics. 129. New York etc.: Springer-Verlag,. xv, 551 p., 144 ill. (1991). ZBL0744.22001.

*Procesi, Claudio, Lie groups. An approach through invariants and representations, Universitext. New York, NY: Springer (ISBN 0-387-26040-4/pbk). xxii, 596 p. (2007). ZBL1154.22001.


For those oriented more towards applications in relativistic and quantum physics:


*

*Barut, Asim O.; Raczka, Ryszard, Theory of group representations and applications. 2nd rev. ed, Warszawa: PWN-Polish Scientific Publishers. XIX, 717 p. (1980). ZBL0471.22021.

*Gel’fand, I. M.; Minlos, R. A.; Shapiro, Z. Ya., Representations of the rotation and Lorentz groups and their applications. Oxford-London-New York-Paris: Pergamon Press. xviii, 366 pp. (1963); Moskva: Gosudarstv. Izdat. Fiz.-Mat. Lit., 368 pp. (1958). ZBL0108.22005.


