@Sharpie
Hi, I think your counterexample works well for NBC, but not for DBC, and such phenomenon originates from the range of indices, $k$ and $l$.
Briefly, only one of $a$ and $b$ contributes to the first positive eigenvalue in NBC, while both of $a$ and $b$ contributes to it in DBC.
For NBC, $k, l \in \mathbb{N}_0$ so that first and second eigenvalue correspond to $(k, l) = (0, 0)$ and $(k, l) = (1, 0)$, respectively.
However, it is not the case for DBC. For DBC, $k, l \in \mathbb{N}$ so that the lowest eigenvalue on small rectangle corresponds to $(k, l) = (1, 1)$, and thus $$\lambda_1 = \pi^{2}\left( {1 \over a^2} + {1 \over b^2} \right)$$. Since the lowest eigenvalue on unit square is then $2\pi^2$, it remains to compare ${1\over a^2} + {1 \over b^2}$ and $2$.
Let's do it. Since the diagonal of rectangle has endpoints on two distinct parallel sides of unit square, $a^2 + b^2 \ge 1$. Henceforth, we get $${1\over a^2} + {1 \over b^2} \ge {1\over a^2b^2}.$$ But since the area of small rectangle is maximized when it is a square, i.e. each vertex of rectangle is a midpoint of each side of unit square, $a^2b^2 \le 1/4$. So we get $${1\over a^2} + {1 \over b^2} \ge {1\over a^2b^2} \ge 4 > 2$$ which concludes the desired monotonicity.