# Detecting a shift in distribution where the distribution is clumpy

I'm responsible for a charity donation site. We're about the change the site design, and we want to know the best way of detecting if the distribution of donations changes after the design. The problem is the data is quite clumpy, particularly around \$5, \$10, \$20, \$25 and \$50 values, with \$15 being relatively rare. There are nonetheless other real values, particularly between \$20 and \$100.

The consequence is that the stdev is three times the mean, so my first approach, a T-Test with some correction for the skew, doesn't seem feasible. If our re-design only has a moderate impact it's unlikely a T-test will be able to detect it with certainty.

Binning into \$5 bins and running a G-Test individually means I can make statements about individual categories, but I'm worried about drawing overall conclusions from such a series of measurements. I've read briefly about using Fisher's method for combining p-values but I'm not sure how to explain the result, particularly as a positive result could mean fewer \$10 donations but more \\$15 donations.

I'm certain that clumpy data is a known phenomenon, but Googling hasn't helped, and my background is CS, not stats. Would anyone know the best way to handle this?

-
–  Steve Huntsman May 17 '10 at 17:24
This is not a "math" answer... but how about starting with some graphs rather than jumping to p values? For instance, after the new design has been up a week, you could make histograms of donations at different levels during each of the 10 weeks preceding the change of design, then making a histogram of the week of the new design. That would give a visual sense of significance. You also may have to factor things like different traffic patterns because of publicity related to a redesign, initial user inexperience, etc. Spending time on exploratory data analysis may make all this clearer. –  Martin M. W. May 17 '10 at 20:29
Thanks for the answers. I was aware of KL divergence having been introduced to it when doing EM, but I was ideally looking for a test that would pass with "95% confidence" or so, like T-Test. KG seems to be a promising approach, am working it through with some sample data now. –  Feenaboccles May 19 '10 at 15:05