For a given $\tau\in S_n$ and for any $j\in[n]$ define the numbers $$a_j:=\chi_{\Delta}(j)-\chi_{\Delta}(j-1)-\chi_{\Delta }(n-j)+\chi_{\Delta}(n-j+1),$$ where $\chi_\Delta:\mathbb{Z}\to\{0,1\}$ denotes the characteristic function of the set $\Delta:=\Delta(\tau)\subset\mathbb{Z}$. Also, with $c:=5n-a_n$, define $$b_j:=a_j-5j+c.$$ We may note right away that since $|a_j|\le2$, the $b_j$ are strictly decreasing, and that $b_n=0$, by the choice of the constant $c$.
For any $E\subset[n]$, for simplicity of notation we put $$ \alpha(E):=\sum_{j\in E} a_j,\qquad \beta(E):=\sum_{j\in E} b_j$$ (so we may think $ \alpha$ and $\beta$ as discrete signed measures supported in $[n]$).
For $i\in[n]$, summing over $j=1,\dots i$ we have
$$ \alpha([i])=\chi_\Delta(i)-\chi_\Delta(n-i).$$
Incidentally, for any $i\in[n]$ we have $i\in\Delta$ if and only if, by definition, $\tau([i])\neq[i]+n-i$ thus also, since $\tau$ is bijective, if and only if $\tau([i]^c)\neq([i]+n-i)^c$, that is $\tau([n-i]+i)\neq [n-i]$ or $\tau^{-1}([n-i])\neq [n-i]+i$, which means $n-i\not\in\Delta^{-1}:=\Delta(\tau^{-1})$. Hence the last formula also writes $$ \alpha([i])=\chi_{\Delta}(i)-\chi_{\Delta^{-1}}(i).$$ Also note that, since $n\not\in\Delta$ $$ \alpha([n])=0,$$ and $$ \alpha([i]+n-i)=- \alpha([n-i]) =-\chi_\Delta(n-i)+\chi_\Delta(i)= \alpha([i]).$$
We proceed showing the inequalities on the arithmetic means.
Case I. Assume $i\in\Delta\setminus\Delta^{-1}$. Then by definition of $\Delta^{-1}$, $\tau^{-1}([i])=[i]+n-i$, so that
$$ {\alpha([i])+ \alpha(\tau^{-1}[i])\over 2i}= {\alpha([i])+ \alpha([i]+n-i)\over2i}={\chi_{\Delta}(i)-\chi_{\Delta^{-1}}(i)\over i}={1\over i}>0, $$ and summing the arithmetic means of $-5j+c$ on the same sets we have plainly $${\beta([i])+ \beta(\tau^{-1}[i])\over 2i}>{\beta([n])\over n}.$$
Case II. Assume $i\in\Delta\cap\Delta^{-1}$. Thus $\tau^{-1}([i])\neq[i]+n-i$ and, just because $b_j$ are strictly decreasing
$${\beta([i])+ \beta(\tau^{-1}[i])\over 2i}>{\beta([i])+ \beta([i]+n-i)\over2i}$$ and since we have $\alpha([i])=\alpha([i]+n-i)=\alpha([n])=0$ because $\chi_{\Delta}(i)=\chi_{\Delta^{-1}}(i)=1$, summing as before the arithmetic means of the affine part of $b_j$, $${\beta([i])+ \beta([i]+n-i)\over2i}={\beta([n])\over n},$$ concluding the proof.