This one is an old classic. I think it is due to Hardy, but you should check Hardy-Littlewood-Polya's "Inequalities".
The answer is $\alpha=2$ as you suggest. You want to prove (after renaming the variables, sorry!) that $$2(\frac{1}{a_1}+\frac{1}{a_2}+\cdots+\frac{1}{a_n})\geq \sum_{k=1}^n \frac{k}{a_1+\cdots+a_k}$$ and to prove this, the hint is to use induction and prove instead the stronger statement $$2(\frac{1}{a_1}+\frac{1}{a_2}+\cdots+\frac{1}{a_n})\geq \sum_{k=1}^n \frac{k}{a_1+\cdots+a_k}+\frac{n^2}{2(a_1+\cdots+a_n)}$$ now it should be clear. If you consider geometric means instead of harmonic means you get a supremum $\alpha=e$, which is Carleman's inequality.