It seems the following.
We have the reverse inequality. Indeed, put $\gamma=\gamma(X,\lambda)$ and $S=\sum_{m \in I} e^{-\gamma x_m}$. The equality $$ \sum_{m \in I} (\lambda-x_m) e^{-\gamma x_m}=0$$ implies $$\lambda S=\sum_{m \in I} \lambda e^{-\gamma x_m}\le \sum_{m \in I\setminus\{1\}} x_m e^{-\gamma x_m}\le \sum_{m \in I\setminus\{1\}} s e^{-\gamma x_m}=s(S-1).$$
It transforms to $S\ge \frac s{s-\lambda}=(1-\lambda/s)^{-1}$.