Given $n \times n$ matrix $M$, you want to find $A,B,C \subset \{1,\dots,n\}$ to maximize 
$$\sum\limits_{i \in A, j \in C} m_{ij} - \sum\limits_{i \in B, j \in C} m_{ij}$$
subject to $A < B < C$ and $|A|=|B|=|C|$.

Introduce binary decision variables $a_i, b_i, c_i \in \{0,1\}$ to indicate whether $i\in A, i\in B, i\in C$, respectively.  Introduce decision variable $d$ to represent the common cardinality. The problem is to maximize
$$\sum_{i=1}^n \sum_{j=1}^n m_{ij} (a_i c_j - b_i c_j)$$
subject to linear constraints
\begin{align}
a_i + b_j &\le 1 &&\text{for $i \ge j$} \\
b_i + c_j &\le 1 &&\text{for $i \ge j$} \\
\sum_i a_i &= d \\
\sum_i b_i &= d \\
\sum_i c_i &= d \\
\end{align}
This is an integer quadratic programming (IQP) problem, but you can linearize by introducing binary variables $u_{ij}$ and $v_{ij}$ to replace the products $a_i c_j$ and $b_i c_j$, respectively.  The resulting integer linear programming (ILP) problem is to maximize
$$\sum_{i=1}^n \sum_{j=1}^n m_{ij} (u_{ij} - v_{ij})$$
subject to linear constraints
\begin{align}
a_i + b_j &\le 1 &&\text{for $i \ge j$} \\
b_i + c_j &\le 1 &&\text{for $i \ge j$} \\
\sum_i a_i &= d \\
\sum_i b_i &= d \\
\sum_i c_i &= d \\
a_i &\ge u_{ij} &&\text{for all $i$ and $j$}\\
c_j &\ge u_{ij} &&\text{for all $i$ and $j$}\\
a_i + c_j - 1 &\le u_{ij} &&\text{for all $i$ and $j$}\\
b_i &\ge v_{ij} &&\text{for all $i$ and $j$}\\
c_j &\ge v_{ij} &&\text{for all $i$ and $j$}\\
b_i + c_j - 1 &\le v_{ij} &&\text{for all $i$ and $j$}
\end{align}