How about these two books for a start?
Tullio Ceccherini-Silberstein, Fabio Scarabotti, and Filippo Tolli. Harmonic analysis on finite groups. Representation theory, Gelfand pairs and Markov chains. Cambridge Studies in Advanced Mathematics 108. Cambridge: Cambridge University Press, 2008.
Persi Diaconis. Group representations in probability and statistics. IMS Lecture Notes-Monograph Series, 11. Hayward, CA: Institute of Mathematical Statistics, 1988.
Referenced added: If you are interested in Lie groups, you could also start with this survey by Emmanuel Breuillard: http://www.math.u-psud.fr/~breuilla/part0gb.pdf