In the usual $\mathsf{ZFC}$, we know that there are $2^\mathfrak{c}$ many subsets of $\mathbb{R}$ that are not (Lebesgue) measurable. On the other hand, the Solovay model also provides us a model of $\mathsf{ZF}$ which has $0$ non-measurable subsets of $\mathbb{R}$.
I would like to know if the following in-between assertion has been established or is currently being researched on:
For any cardinal $\kappa < 2^\mathfrak{c}$, is it consistent with $\mathsf{ZF}$ that there are exactly $\kappa$ non-measurable subset of $\mathbb{R}$?
If this is false, can we classify the cardinals in which the assertion above holds?