Having screwed up the answer by getting the wrong answer for a really simple calculation the first time, I'm now going to try to redeem myself.

First, to make things easier, let each point be present with probability $h/(mn)$, and let these random variables be independent. This won't change the probabilities much.

You can calculate explicitly the expected number of squares. The expected number of squares is ${m \choose 2}{n \choose 2}(h/mn)^4$, since there are ${m \choose 2}{n \choose 2}$ possible squares, and each point is present with probability $h/mn$. This is roughly $1/4 (h^2/mn)^2$ (for large $mn$, and fixed $h^2/mn$).

Now, let's calculate the expected number of six-cycles. You can assume a $2k$-cycle lies in $k$ different columns and $k$ different rows (otherwise there is a smaller cycle), so we have ${m \choose 3}{n \choose 3}\alpha_3 (h/(mn))^6$ ways of getting a six-cycle. Here, $\alpha_3$ is the ways of getting a six-cycle in a 3 by 3 square, and it is easy to check that there are six of them.

There are four ways that it can look like this:

```
1.1
.11
11.
```

and two ways it can look like this:

```
11.
1.1
.11
```

Now, if we can compute $\alpha_k$, where $\alpha_k$ is the number of ways we can get a minimal $2k$-cycle in a $k\times k$ grid, we can get an infinite series for the expected number of cycles. For $\alpha_k$, we need to find a cycle which covers every row twice and every column twice. Let $\pi_1$ be the order that we visit the rows, and $\pi_2$ the order we visit the columns. We can assume that $\pi_1$ starts with $1$, and we can traverse each cycle in $2$ directions. This means that there are $(k-1)!$ possible $\pi_1$, $k!$ possible $\pi_2$, and we have to divide by $2$ because otherwise we count each cycle twice. Thus, the expected number of cycles when $m,n\rightarrow \infty$ is
$$ \sum_{k=2}^\infty \frac{1}{2}(k-1)! k!{m \choose k}{n \choose k}\left(\frac{h}{mn}\right)^{2k},$$
which simplifies in the limit to
$$ \sum_{k=2}^\infty \frac{1}{2k}\left(\frac{h^2}{mn}\right)^k,$$
or
$$-1/2 \ln(1 - x) - x/2, \mathrm{\ \ where\ }x=h^2/(mn).$$
If objects are distributed with a Poisson distribution with expectation $r$, then the probability that there is an object is $1-e^{-r}$. Thus, if we assume that the closed paths are distributed according to the Poisson distribution, the probability that a closed path exists is
$$1-(1-x)^{1/2}e^{x/2}.$$
As Kevin Costello remarks in the comments, you may be able to use Janson's theorem (also covered in Alon and Spencer's book) to prove that these closed paths obey Poisson statistics.

And below, another argument that the probability of a closed path rises to 1 when $h^2 =mn$, salvaged from my first attempt at solving the problem.

Let's start at a single point $P_0$, and look at the tree of paths that results. The tree of paths is obtained by looking for points in the same row as $P_0$, then points in the same column as these points, and then rows, etc. (Also do the same procedure, but starting with columns.) If any two points in this tree are equal (that is, there are two distinct paths leading from the starting point to the same point) then there is a closed path. So we want to know the probability that this tree of paths is fairly large. In particular, if the tree of paths has size $\sqrt{h}$, then by the birthday paradox there should be a reasonable chance that two of these points are equal.

But, assuming all the points are different, this tree of paths is a Galton-Watson branching process, and we can calculate the probability that it's large. (You can calculate lots of properties of Galton-Watson branching processes.) In particular, if the expected number of children is greater than 1, there is a finite probability that the tree is infinite. Using this is a rough criterion, you will start to get closed paths when $mn \approx h^2$. You should be able to a much better approximation by analyzing the associated Galton-Watson branching process more carefully.

The Galton-Watson tree has two branches. On the first branch, the number of children on odd levels are distributed approximately as a Poisson process with expectation $h/m$, and the number of children on even levels as a Poisson process with expectation $h/n$. On the second branch, distributions on the odd and even levels are reversed.