The first few lines of this post is based on this lecture notes, but similar expositions can be found in other physics books such as Peskin & Schroeder's book.
On chapter 8 of the linked notes, the author is interested on finding all representations of the Lorentz group which make the physics of the system invariant.
Let me sketch the main points of his exposition. A small Lorentz transformation is: \begin{eqnarray} \Lambda_{\nu}^{\mu} = \delta_{\nu}^{\mu} + \omega_{\nu}^{\mu} \tag{1}\label{1} \end{eqnarray} where the parameters of the transformation satisfy $\omega_{\nu}^{\mu} = -\omega_{\mu}^{\nu}$. Any $n\times n$ matrix representation of such transformation on fields $M_{ab}(\Lambda_{\nu}^{\mu}) = M_{ab}(\delta_{\mu}^{\nu}+\omega_{\nu}^{\mu})$ is expanded: \begin{eqnarray} M_{ab}(\Lambda_{\nu}^{\mu}) = M_{ab}(\delta_{\nu}^{\mu}) + \sum_{\mu< \nu}\omega_{\mu\nu}\frac{\partial M_{ab}(\Lambda)}{\omega_{\mu\nu}} + O(\omega^{2}) \tag{2}\label{2} \end{eqnarray} Then, define the generators of the representation: \begin{eqnarray} (J^{\mu\nu}_{M})_{ab} := i\frac{\partial M_{ab}(\Lambda)}{\omega_{\mu\nu}} \tag{3}\label{3} \end{eqnarray} and exponentials of those $J$ are supposed to generate all Lorentz transformations.
Now, the above scenario seems very non-rigorous. To begin with, a representation of a group $G$ is a homomorphism $\rho : G \to GL(V)$ where $V$ is a vector space and $GL(V)$ the group of all linear transformations from $V$ to itself (the product being the composite of functions). Thus, I think that by "All representations of the Lorentz group" physicists mean something like every element of $GL(V)$ for which the theory is invariant.
In any case, I'd like to know if there exists a mathematical precise formulation of the above. I have no idea what even to look for on the internet, so any help is appreciated. Could someone explain the details of it or provide good references, if possible? In special, I'd like to understand what are these $J$ given by (\ref{3}), what they generate and so on. More importantly, for the scalar representation, $J$ should be: \begin{eqnarray} J^{\mu\nu} = -i(x^{\mu}\partial^{\nu}-x^{\nu}\partial^{\mu}) \tag{4}\label{4} \end{eqnarray} and this seems to be of special interest in the theory, understanding (\ref{4}) is very important to me.
