I have used mathematica to test following equation is true
\begin{equation}
\sum_{a,b=0}^{m-1} \left[\frac{a+b n}{m}\right] = \frac{n m^2}{2} -\frac{nm}{2}
\end{equation}
where $[x]$ is the floor function in mathematica

Do you have analytic derivation that how to get the RHS?

Thanks