I'll try to show that we can make a square such that all rows and columns give zero sum. (Where "sum" is meant in the sense described in the question.) I.e., this is the answer to the stronger variant.^{1}

The description is a bit informal, but I hope it could be clear how the construction goes.

We will proceed by induction and choose values $j(k,l)$ with $-N\le k,l \le N$. I.e., after $N$ steps the square of the size $(2N+1)\times(2N+1)$ will be filled.
Moreover, will do it in such fashion that the row sums and column sums of the square are zeroes. (And in such way that no numbers in our table repeat.)

**Base step.** Let us start by putting $j(0,0)=0$.

In the first step of induction we want to add some other numbers into this three by three square
$$
\begin{array}{|c|c|c|}
\hline
\hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline
\hphantom{0} & 0 & \hphantom{0} \\\hline
\hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline
\end{array}
$$

Now we simply choose two distinct positive integers $a$, $b$ and add also $-b$, $-a$ in the opposite positions so that we get zeroes in the middle row and the middle column.
$$
\begin{array}{|c|c|c|}
\hline
\hphantom{-a} & a & \hphantom{-a} \\\hline
b & 0 & -b \\\hline
\hphantom{-a} &-a & \hphantom{-a} \\\hline
\end{array}
$$

We simply choose any integer $x>\max(a,b)$. If we then add $x$ to the topleft corner, we have only one possibility what to do in the other positions in order to get sum equal to zero

$$
\begin{array}{|c|c|c|}
\hline
x & a & -a-x \\\hline
b & \hphantom{a+}0\hphantom{+b} & -b \\\hline
-b-x &-a & a+b+x \\\hline
\end{array}
$$

The condition $x>\max(a,b)$ implies that all numbers in this table are distinct.

*Inductive step.* We assume that we already have a square where row/columnns add up to zero. We want to add two more rows (at the top and at the bottom) and two more columns (left and right).

The construction in the inductive step will be somewhat similar to what we did in the base step.

We already have a "zero square" in the middle.
$$
\begin{array}{|c|ccc|c|}
\hline
\hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline
\hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\
\hphantom{0} & \hphantom{0} & 0 & \hphantom{0} & \hphantom{0} \\
\hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline
\hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline
\end{array}
$$

We first add numbers above/below the square and also to the left and to the right. (I.e., only the four corners will be missing.)
We will do this in such way that above the square we put some positive integers that neither these numbers, nor their opposites, have been used so far. And below the square we put their opposites. Similarly on the left and right side. After this the row and column sums are zeroes, with the possible exception of the first and last row/column - where we're going to add the missing values.
Let us denote the sum of all numbers added "above" as $A$ and the sum of all numbers added on the left as $B$.
We can also require that $A\ne B$. (If needed, we simply modify one of the numbers on the left.)
$$
\begin{array}{|c|ccc|c|}
\hline
\hphantom{0} & \hphantom{0} & A & \hphantom{0} & \hphantom{0} \\\hline
\hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\
B & \hphantom{0} & 0 & \hphantom{0} & -B \\
\hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline
\hphantom{0} & \hphantom{0} & -A & \hphantom{0} & \hphantom{0} \\\hline
\end{array}
$$
Again we choose some $x$ which is larger than the absolute values of the numbers we have used so far. (This will ensure that the numbers in the corners will be distinct from the existing ones.)
The we have only one possibility what to add in the remaining positions.
$$
\begin{array}{|c|ccc|c|}
\hline
x & \hphantom{0} & A & \hphantom{0} & -x-A \\\hline
\hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\
B & \hphantom{0} & 0 & \hphantom{0} & -B \\
\hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} & \hphantom{0} \\\hline
-B-x & \hphantom{0} & -A & \hphantom{0} & x+A+B \\\hline
\end{array}
$$

Continuing in this way, we fill the whole $\mathbb Z\times\mathbb Z$ table in a way which fulfills the requirements stated in the question.

^{1}Ben Barber suggested some approach in the comments to the original question. I do not understand those comments well enough to be able to judge whether the solutions proposed there is similar to this one. However, one of the comments suggests that we are in fact able to get a bijection - which seems more difficult (and more interesting).

Antoine Salomon, "Infinite sized magic square" the American Mathematical Monthly (January 2014)(freely available at Hal). There is also a discussion elsewhere on this site about infinite magic squares." $\endgroup$ – YCor Oct 19 '19 at 9:59