You can also create degree-2 vertices when you contract an edge. For example, in a triangular prism, contracting an edge from a triangle creates a degree-2 vertex (after suppressing parallel edges). In fact, the triangular prism is a counterexample, given your definitions of $G -e$ and $G / e$. Maybe you also want to supress degree-2 vertices when contracting?

I think the situation you describe is impossible since the fourth set always has sum at least $S/4$ while the first two always each have sum at most $S/4$. Thus, the largest element from the fourth set will always be at least as large as the smallest element from one of the first two sets. Also, I think we might not be done without performing the last swap. For example, $d_1=d_2=d_3=-0.33$ and $d_4=0.99$ is possible (in the first case). Feel free to send me an email if it is still unclear (my email can be easily found online).

I added more details. Please let me know if it is clear. I think I can also prove the full theorem (without any special assumptions). I will write it up here when I have time.

Thanks! Regarding your question, $\ell_1$ is the minimum of $Y_1$ but the $(n_1+1)$th smallest element of $X_1$ is a candidate for $\ell_1$, so the inequalities become even better if $\ell_1$ is not the $(n_1+1)$th smallest element of $X_1$.