The representation theory of finite semigroups is an interesting blend of group representation theory and the representation theory of finite dimensional algebras. The subject is both old, going back to A.H. Clifford (from Clifford theory in group representation theory), and at the same time is in its infancy.
The reason why semigroup representation theory is not so well studied, in my opinion, lies in its origins. A description of the simple modules for a finite semigroup was given by work of Clifford, Munn and Ponizovsky in the forties and fifties. It was further clarified by Rhodes and Zalcstein and by Lallement and Petrich in the sixties. Roughly speaking the main theorem states that all irreducible representations of a finite semigroup can be constructed from irreducible representations of associated finite groups in a very explicit way. Sadly, this beautiful work was written up using very heavily the structure theory of finite semigroups, which is not widely known, and so the literature is virtually inaccessible to nonspecialists. The approach used here foreshadows the development of stratified and quasihereditary algebras by Cline, Parshall and Scott. In fact, in 1972, Nico computed a bound on the global dimension of the algebra of a finite von Neumann regula semigroup by finding a sequence of heredity ideals and discovering the bound it gives on global dimension years before the notion of heredity ideal was invented.
The character table of a finite semigroup was investigated in the sixties and seventies and shown to be invertible (although it is not orthogonal like in the group case). A method for writing a class function as a linear combination of irreducible characters was given that amalgamated the group situation with Möbius inversion in posets.
Progress on finite semigroup representation theory then more of less stalled for a number of years. I believe this was for two main reasons.
- There was a lack of ready-made applications.
- Semigroup algebras are almost never semisimple and the modern representation theory of quivers, etc. were only invented on the seventies. By then finite semigroup theorists were interested in other problems and they were mostly unaware of developments in the representation theory of finite dimensional algebras.
In the eighties and early nineties, there was some renewed investigation of the representation theory of finite semigroups, mostly due to Putcha, Okninski, Renner and their collaborators. In particular, connections with quivers and quasihereditary algebras and other aspects of modern representation theory were made.
The past decade has seen a renaissance in the subject of semigroup representation theory, spurred on by probabilists and algebraic combinatorialists. Bidigare, Hanlon, Rockmore, Diaconis and Brown, to name a few, have shown that a number of random walks are much more easily analyzed using semigroup representation than using group representation theory. For instance, it is nearly trivial to compute eigenvalues for the riffle shuffle and the top-to-random shuffle using semigroup theory. It is more difficult to use the representation theory of the symmetric group. Moreover, the diagonalizabilty of these walks is not explained by group theory, but it is explained by semigroup theory.
Also Bidigare's observation that Solomon's descent algebra associated to a finite Coxeter group is a subalgebra of a hyperplane face semigroup algebra has been important to people in algebraic combinatorics. There are also applications of semigroup theory to automata theory in particular in connection to the notorious Cerny conjecture on synchronizing automata.
In the last year, a half-dozen papers on semigroup representation theory have appeared on the ArXiv, many by nonsemigroup theorists. I expect that the trend will continue. We now know how to compute the quiver for a large class of finite semigroups, describe in semigroup theoretic terms projective indecomposable modules and for some classes we have techniques for computing global dimension. Semigroups with basic algebras over a given field have been described.
What is needed is a book covering all this for the general public!
Edit. Our new paper gives a close connection between monoid representation theory, poset topology and Leray numbers of simplicial complexes with classifying spaces of small categories thrown in. If browsing this paper doesn't convince you that monoid representation theory has something to it, then I don't know what will.
Edit. (2/18/14) Since this question just got bumped, let me add the new paper http://arxiv.org/abs/1401.4250 which gives a general introduction to Markov chains and semigroup representation theory and new examples.