I think the book [Engineering Applications of Noncommutative Harmonic Analysis](http://books.google.com/books/about/Engineering_Applications_of_Noncommutati.html?id=o9wArx_zIsEC) by Chirikjian and Kyatkin might be exactly what you are looking for.  Although I haven't read very much of it, the sections that I have read are very nice and seem mathematically rigorous, yet geared toward applications.

Also, there are a couple of excellent books by Myoung An and Richard Tolimieri about harmonic analysis over finite groups.  The first is called [Time-Frequency Representations](http://www.amazon.com/Time-Frequency-Representations-Numerical-Harmonic-Analysis/dp/0817639187/), and it's about harmonic analysis over finite **abelian** groups with applications to audio signal processing.  The second is called [Group Filters and Image Processing](http://www.psypher.org/books-GFIP.html), and it's about harmonic analysis over finite **nonabelian** groups with applications to image processing.  I can't say enough about these books.  The mathematical presentation is rigorous and elegant, and the applied examples are very explicit, including Matlab code and demonstrating how the authors have applied the techniques in work they have done for defense contractors.  An overview of some of the material presented in these two book appears in [this paper](http://prometheus-us.com/asi/algebra2003/papers/tolimieri.pdf).  

I highly recommend the books by Tolimieri and An -- especially "Time-Frequency Representations" -- to anyone who wants to learn signal processing the "right" way.  (Incidentally, these authors have a new book called [Ideal Sequence Design in Time-Frequency Space](http://www.amazon.com/Ideal-Sequence-Design-Time-Frequency-Space/dp/0817647376/), but I have not read it.)