2
$\begingroup$

A relatively simple question, but I cannot seem to find anything relevant to this. In fact, I'm not even sure of "unbalanced design" is even the right terminology here, but it was suggested to me.

The problem may be described as follows. Let's say you have a partition of your population into some groups - A, B and C. These groups contain 100, 200 and 500 individuals respectively. You don't know that - you only know the relative sizes (so, B is 2x the size of A, C is 5x the size of A, etc.). Furthermore, you have three random samples, of sizes 50, 20 and 100 from each of these.

How does one construct a linear regression here (nothing fancy) that combines the data from all three samples?

$\endgroup$
2
  • $\begingroup$ Did you try asking your question at stats.stackexchange.com ? You might get more interest there (I notice that you asked your question one hour ago already and there hasn't been any activity since). MO does not seem to have many statisticians. $\endgroup$ Jan 27, 2011 at 23:22
  • $\begingroup$ Thanks, didn't know about it; I guess I'll wait and see first if anything surfaces here. $\endgroup$
    – ALX713078
    Jan 27, 2011 at 23:44

2 Answers 2

1
$\begingroup$

I think that you should use a weighted regression here. Suppose that you have $n$ data points, for each you see the predictor vector $\vec{x_i}$ and response variable $y_i$. Suppose moreover that these points belong to $m$ different sub-populations, with proportions $\alpha_1,..,\alpha_m$ summing to one (in your case you have 3 sub-populations with proportions $1/8, 2/8, 5/8$) and observed sample sizes $n_1, .., n_m$ summing to $n$ (in your case $50, 20, 100$). Then instead of performing ordinary least-square regression, i.e. finding $\vec{\beta}$ minimizing the sum of squares: $\sum_i (y_i - \vec{x_i} \vec{\beta})^2$, you should minimize: $\sum_i w_i (y_i - \vec{x_i} \vec{\beta})^2$. The weights $w_i$ are determined as follows: If the $i$-th observation was taken from the $j$-th sup-population, then set $w_i = \alpha_j / n_j$. This guarantees that the total weight in the regression cost function of each sub-population will be proportional to it's fraction in the population. For example, in your case each observation from $A$ should be assigned a weight $\frac{1/8}{50}$.

$\endgroup$
1
$\begingroup$

The class imbalance you describe isn't that problematic. Just use regression as you would normally do. In cases where the imbalance ratio is around 1:10000 you would need to use different methods to deal with the problem such as: Downsampling, upsampling, SMOTE, etc.

Take a look at: http://appliedpredictivemodeling.com/ There is a complete chapter regarding the problem. But as I say, in your case it's not needed.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.