You might want to look at Section 3.1 of "Group Theory and Physics" by Shlomo Sternberg, Cambridge Univ Press, 1994. This explains, through a simple example, how (in Sternberg's words) "molecular spectroscopy is an application of Schur's lemma". The argument is elementary in nature. The last chapter of the book by James & Liebeck (Representations and Characters of Groups 2e, Cambridge Univ Press, 2001) is a longer exposition of the same idea. I notice another post here about work by Diaconis - Diaconis has a book called "Group Representations in Probability and Statistics" which is available for free download. See the link at
http://www.math.columbia.edu/~khovanov/resources/
This page has links to dozens of useful articles. Also, there is the book "Unitary Group Representations in Physics, Probability and Number theory" by George W Mackey (Benjamin/Cummings Publ Co, 1978). This is more advanced than the others though. For applications to quantum chemistry there is (amongst many) "Chemical Applications of Group Theory" by F Albert Cotton, published by John Wiley. If you want to see how Section 2.7 of Serre's book is actually used in practice by chemists, see Chapter 6 in Cotton's book.

