Infinite dimensional unitary representations of SU(2) for non-half-integer j? The finite dimensional irreducible unitary representations of $SU(2)$ are labelled by $j$ which needs to be half-integer, the dimension of the representation is $2j+1$. This is well-known, all is good.
If we do not require finite dimension for the representation, is it possible to make sense of representations with an arbitrary real number $j$? They will, presumably, be infinite dimensional but hopefully still unitary.
In the half-integer case when represented on at most $2j$ degree holomorphic polynomials, the 3 basis elements of the Lie-algebra in representation $j$ act as
$e_1 = \frac{1-z^2}{2}\frac{d}{dz} + jz$
$e_2 = \frac{1+z^2}{2i}\frac{d}{dz} + ijz$
$e_3 = -z\frac{d}{dz} + j$
Clearly, if $j$ is not half-integer and we start from $f(z) = z$ and start acting on it with $e_i$, it will generate an infinite dimensional space.
This kinda gives me the feeling that perhaps non-half-integer $j$ representations are still meaningful and are infinite dimensional. But I'm not sure, is this really the case or something will go wrong?
Basically what I'm asking is whether analytic continuation in $j$ makes any sense.
 A: You have to distinguish between representations of a compact group $SU(2)$ and of its Lie algebra $su(2)$, which can be complexified to $sl(2,\mathbb{C}).$ By Peter-Weyl theorem, every irreducible representation of the group $SU(2)$ is finite-dimensional. However, you can let Lie algebra $sl(2)$ act on various infinite-dimensional spaces by differential operators, e.g.
$$e\to z^2 d/dz, h\to zd/dz, f\to d/dz,$$ 
$$\text{ where as usual}\quad 
e=\begin{pmatrix} 0 & 1\\ 0 & 0\end{pmatrix},
h=\begin{pmatrix} 1 & 0\\ 0 & -1\end{pmatrix},
f=\begin{pmatrix} 0 & 0\\ 1 & 0\end{pmatrix},
$$
$$\text{acts on }\quad V=z^\mu\mathbb{C}[z,z^{-1}]$$ (this is probably equivalent to your formulas after some change of coordinates).  They will not exponentiate to an action of $SU(2)$.
Nonetheless, as Scott indicated in the comments, some of these representations do arise from bona fide representations of the group $SL(2,\mathbb{R})$, which has the same complexified Lie algebra. This group is noncompact and some of them are unitary and some are not. You can use algebraic methods to describe a large class of representations of $SL(2,\mathbb{R})$ in this way. I've omitted some technical details, but all this is beautifully explained in Howe and Tan's book.
A: I just wrote this answer on your last question. To summarize, you can't have an irreducible representation of a compact group that is infinite dimensional, unless the representation space is very exotic.
By the Peter-Weyl Theorem, all irreducible Hilbert space representations of a compact group (e.g. SU(2)) are finite dimensional. Thus, any infinite dimensional Hilbert space representation will be reducible.
What can be said for non-Hilbert space representations? Given a compact group G acting irreducibly (and continuously) on a locally convex topological vector space V, you can inject V into L2(G) by sending v in V to the function cv(g)=〈g⋅v,v'〉 where v' is basically any nonzero element of the continuous dual of V (this is where local convexity is used). (Note that the irreducibility of V implies the injectivity of the map.)
Thus V is finite dimensional: its image is not necessarily closed in L2(G), but, since V is irreducible, its image has to lie within a single irreducible component of L2(G), which are all finite dimensional.
So to find a representation that is infinite dimensional and irreducible, you'd have to look at non-locally convex vector spaces (actually, you need the dual space to not separate points). Like Lp with p<1.
