# How to construct a linear regression model with an unbalanced sample design?

A relatively simple question, but I cannot seem to find anything relevant to this. In fact, I'm not even sure of "unbalanced design" is even the right terminology here, but it was suggested to me.

The problem may be described as follows. Let's say you have a partition of your population into some groups - A, B and C. These groups contain 100, 200 and 500 individuals respectively. You don't know that - you only know the relative sizes (so, B is 2x the size of A, C is 5x the size of A, etc.). Furthermore, you have three random samples, of sizes 50, 20 and 100 from each of these.

How does one construct a linear regression here (nothing fancy) that combines the data from all three samples?

-
Did you try asking your question at stats.stackexchange.com ? You might get more interest there (I notice that you asked your question one hour ago already and there hasn't been any activity since). MO does not seem to have many statisticians. – Thierry Zell Jan 27 '11 at 23:22
Thanks, didn't know about it; I guess I'll wait and see first if anything surfaces here. – ALX713078 Jan 27 '11 at 23:44

I think that you should use a weighted regression here. Suppose that you have $n$ data points, for each you see the predictor vector $\vec{x_i}$ and response variable $y_i$. Suppose moreover that these points belong to $m$ different sub-populations, with proportions $\alpha_1,..,\alpha_m$ summing to one (in your case you have 3 sub-populations with proportions $1/8, 2/8, 5/8$) and observed sample sizes $n_1, .., n_m$ summing to $n$ (in your case $50, 20, 100$). Then instead of performing ordinary least-square regression, i.e. finding $\vec{\beta}$ minimizing the sum of squares: $\sum_i (y_i - \vec{x_i} \vec{\beta})^2$, you should minimize: $\sum_i w_i (y_i - \vec{x_i} \vec{\beta})^2$. The weights $w_i$ are determined as follows: If the $i$-th observation was taken from the $j$-th sup-population, then set $w_i = \alpha_j / n_j$. This guarantees that the total weight in the regression cost function of each sub-population will be proportional to it's fraction in the population. For example, in your case each observation from $A$ should be assigned a weight $\frac{1/8}{50}$.