I apologise for the long-windedness of this question.

Let $a$ be a positive real constant and let $d(n)$ denote the number of divisors of $n.$ Define $$ S_a(x)=\sum_{n\leq x} d(n)^a. $$ For $a=1,$ the following is well known $$ S_1(x)=\sum_{n\leq x} d(n)=x \log x + (2 \gamma -1) x +{\cal O}(\sqrt{x}) $$ while for more general $a$, one has $S_a(x) \sim C(a) x (\log x)^{2^a -1}$ where $$ C(a) = \Gamma(2^a)^{-1} \prod_p \left( 1 - \frac{1}{p} \right)^{2^a} \left( \sum_{k \geq 0} \frac{(k+1)^a}{p^k}\right). $$ More accurate estimates are available via the Selberg-Delange method, though the details get quite technical.

My question is the following. Let the subset $A$ be defined by $A\subset \{1,2,\ldots,x\},$ (assume $x$ is integer or use the floor function) with $$\#A\geq \frac{x}{2}+ c \frac{\log x}{\log \log x}:=Z(x).$$ Now define $$ S^{A}_{a}(x)=\sum_{n\leq x:n \in A} d(n)^a. $$ I want to lower bound this sum, as $A$ varies subject to the size condition, i.e., to derive a lower bound to $$ M:=\min \left\{ S_a^A(x): A \subset \{1,\ldots,x\}, \#A =Z(x) \right\}. $$ For $a=1,$ I hope a lower bound of the form $M\gg x \log x$ may be possible if the Poisson approximation is good enough. Essentially this would say that half the points achieve a constant factor of the full sum, even when restricted to those points with the fewest number of divisors.

From a Poisson approximation point of view, it would seem that approximately half the points $n\in \{1,\ldots,x\}$ have $\omega(n)\leq \log\log n,$ with the relevant Poisson distribution having mean $\log\log n.$

I am unsure if the known techniques are strong enough to address such a delicate size specification, compared to say $\lceil x/2 \rceil,$ but some comments related to Selberg-Delange have a discussion of how a multiplicative $O(1+(\log x)^{-c'})$ factor can be shown for various sums, which may imply that it is indeed possible.