Lets make the expansion of the next sum around $h=1$
$$
\left ( \prod_{n=1}^{N}  \ \sum_{l_n=0}^{N_f} \ h^{2 l_n} \
 \binom{N_f}{l_n} \right ) \ \underset{1\le i,j\le N}{\det} \binom{N_f}{l_i-i+j}   =|h^{2 l_n} =1+2 l_n \log h + ...  |
$$
$$
=  \underset{1\le i,j\le N}{\det} \binom{N_f^2}{N_f-i+j} + \log h \sum_{k=1}^N  \underset{1\le i,j\le N}{\det}  (N_f+ k-j) \binom{N_f^2}{N_f -i+j} +
$$
$$
+ (\log h)^2 \left \{ \sum_{k=1}^N   \underset{1\le i,j\le N}{\det}  \frac{(N_f+ k-j)(N_f^2+(1-N_f)(k-j))}{2 N_f -1} \binom{N_f^2}{N_f-i+j} \right. +
$$
$$
\left. +\sum_{k, l=1}^N  \underset{1\le i,j\le N}{\det} (N_f+ k-j)  (N_f+ l-j)  \binom{N_f^2}{N_f-i+j} \right \} + 
$$
$$
+ (\log h)^3\left \{\sum_{k=1}^N  \underset{1\le i,j\le N}{\det}  \frac{ (N_f+k-j)^2 \left(N_f(N_f+1)+ (2- N_f)(k-j)\right)}{3(2 N_f-1)} \binom{N_f^2}{N_f-i+j}  + ... \right \}+ O( (\log h)^4)
$$
Using
$$
\sum_{s=0}^{N_f} \ \binom{N_f}{s} \binom{N_f}{s-k +l} = \binom{2 N_f}{N_f-k+l} 
$$
\begin{eqnarray}
  \underset{1\le k,l \le N}{\det} \ \binom{2 N_f}{N_f-k+l} = \frac{ G[N+ 2 N_f+1] G[N+1] G[N_f+1]^2 }
{ G[2 N_f+1] G[N+ N_f+1]^2 }\ = Z_0  
\end{eqnarray}
and first correction is
$$
\sum_{k=1}^N  \underset{1\le i,j\le N}{\det}  (N_f+ k-j) \binom{N_f^2}{N_f-i+j}  = N_f N \, Z_0
$$
because of
$$
\sum_{k=1}^N  \underset{1\le i,j\le N}{\det}  ( k-j) \binom{N_f^2}{N_f-i+j} = \epsilon^{j_1 .. j_N} [(1- j_1)+... +(N- j_N)]\binom{N_f^2}{N_f-1+j_1}  ... \binom{N_f^2}{N_f-N+j_N}   = 0 \ .
$$
But at next terms we get in trouble (looking at $- 2k\cdot j$ terms)
$$
\sum_{k=1}^N  \underset{1\le i,j\le N}{\det}  ( k-j)^2 \binom{N_f^2}{N_f-i+j} = \epsilon^{j_1 .. j_N} \left \{ [(1+ j_1^2)+... +(N^2+ j_N^2)]  + \right.
$$
$$
\left. 
- 2( j_1+ 2 j_2+ ...+  N j_N) \right  \} \binom{N_f^2}{N_f-1+j_1}  ... \binom{N_f^2}{N_f-N+j_N}=
2 \frac{N(N+1)(2 N+1)}{6} Z_0 + ???
$$
How is to get determinants of this type?