Take the 2-minute tour ×
MathOverflow is a question and answer site for professional mathematicians. It's 100% free, no registration required.

A relatively simple question, but I cannot seem to find anything relevant to this. In fact, I'm not even sure of "unbalanced design" is even the right terminology here, but it was suggested to me.

The problem may be described as follows. Let's say you have a partition of your population into some groups - A, B and C. These groups contain 100, 200 and 500 individuals respectively. You don't know that - you only know the relative sizes (so, B is 2x the size of A, C is 5x the size of A, etc.). Furthermore, you have three random samples, of sizes 50, 20 and 100 from each of these.

How does one construct a linear regression here (nothing fancy) that combines the data from all three samples?

share|improve this question
Did you try asking your question at stats.stackexchange.com ? You might get more interest there (I notice that you asked your question one hour ago already and there hasn't been any activity since). MO does not seem to have many statisticians. –  Thierry Zell Jan 27 '11 at 23:22
Thanks, didn't know about it; I guess I'll wait and see first if anything surfaces here. –  ALX713078 Jan 27 '11 at 23:44

2 Answers 2

I think that you should use a weighted regression here. Suppose that you have $n$ data points, for each you see the predictor vector $\vec{x_i}$ and response variable $y_i$. Suppose moreover that these points belong to $m$ different sub-populations, with proportions $\alpha_1,..,\alpha_m$ summing to one (in your case you have 3 sub-populations with proportions $1/8, 2/8, 5/8$) and observed sample sizes $n_1, .., n_m$ summing to $n$ (in your case $50, 20, 100$). Then instead of performing ordinary least-square regression, i.e. finding $\vec{\beta}$ minimizing the sum of squares: $\sum_i (y_i - \vec{x_i} \vec{\beta})^2$, you should minimize: $\sum_i w_i (y_i - \vec{x_i} \vec{\beta})^2$. The weights $w_i$ are determined as follows: If the $i$-th observation was taken from the $j$-th sup-population, then set $w_i = \alpha_j / n_j$. This guarantees that the total weight in the regression cost function of each sub-population will be proportional to it's fraction in the population. For example, in your case each observation from $A$ should be assigned a weight $\frac{1/8}{50}$.

share|improve this answer

The class imbalance you describe isn't that problematic. Just use regression as you would normally do. In cases where the imbalance ratio is around 1:10000 you would need to use different methods to deal with the problem such as: Downsampling, upsampling, SMOTE, etc.

Take a look at: http://appliedpredictivemodeling.com/ There is a complete chapter regarding the problem. But as I say, in your case it's not needed.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.