Since $m$ is a dummy variable (*i.e.* a bound variable) and $n,n'$ are "real" variables (*i.e.* they are free) perhaps we should rewrite the problem accordingly as
$``$compute the following
$$ f(y,z) = \#\left\lbrace (x_1,... , x_y )\mid x_1 + ... + x_y = m,\ x_i \in \mathbb{N},\ m \leq yz ,\ i < j \implies x_i \leq x_j \right\rbrace."$$
We can let $f'$ be the number if we omit the condition $i < j \implies x_i \leq x_j$ (*i.e.* $f$ corresponds to partitions and $f'$ corresponds to compositions). Intuition tells us that $f$ will likely have a closed form solution and $f'$ will probably only have solution expressible by generating functions (which are perfect for asymptotics). Let us confirm our intuitions:

**Compositions (***i.e.* $f'$)

It is straightforward to see that if we let
$$g(y,m) = \#\left\lbrace (x_1,..., x_y ) \mid x_1 + ... + x_y = m, \ x_i \in \mathbb{N}\right\rbrace,$$
then we have the identity
$$f'(y,z) = \sum_{m=0}^{yz}g(y,m).$$
It is well known, see the famous stars and bars method in feller's introduction to probability theory, that
$$g(y,m)= \binom{y+m-1}{m},$$

so that $$f'(y,z) = \sum_{m=0}^{yz}\binom{y+m-1}{m}=
\binom{yz+y}{yz},$$
(or, using your notation, $f'(n,n')=\binom{nn'+n}{nn'}$)

where the last identity can be derived as a consequence of Chu Shih-Chieh's identity, see example 2.5.1 of Chuan-Chong & Khee-Meng's text on combinatorics. I also strongly recommend taking a peek at Flajolet & Sedgewick's text on analytic combinatorics for asymptotics and the more abstract symbolic method/species style which is necessary for the more difficult analysis of $f$.

**Partitions (***i.e.* $f$)

It is straightforward to see that if we let
$$g(y,m) = \#\left\lbrace (x_1,... , x_y ) \mid x_1 + ... + x_y = m, \ x_i \in \mathbb{N}, \ i < j \implies x_i \leq x_j\right\rbrace,$$
then we have the identity
$$f(y,z) = \sum_{m=0}^{yz}g(y,m).$$

It is well known that if we define the generating function $\mathcal{G}_y$ as
$$\mathcal{G}_y(x) = \sum_{m \in \mathbb{N}} g(y,m) x^m,$$
then we have that
$$\mathcal{G}_y(x) = \prod_{k=1}^{y}\frac{1}{1-x^k},$$
see Flajolet & Sedgewick's text on analytic combinatorics or Andrews elementary text on partitions. One way to see this is to notice the following famous theorem attributed to Euler

The number of partitions of a number $n$ into at most $l$ parts is equal to the number of partitions of a number $n$ into parts all bounded by $l$

and the result follows by elementary generating function magic. Finally, theorem 5.1.1 of Chuan-Chong & Khee-Meng's text on combinatorics states that
$$\mathcal{A}(x) = \sum_{n \in \mathbb{N}} a_n x^n \implies \frac{1}{1-x}\mathcal{A}(x) = \sum_{n \in \mathbb{N}} \left(\sum_{k \leq n} a_k \right) x^n;
$$
therefore, if we define the generating function $\mathcal{F}_y$ as
$$\mathcal{F}_{y}(x) = \sum_{n \in \mathbb{N}} f(y,n) x^n,$$

then we have that $$\mathcal{F}_{y}(x) = \frac{1}{1-x}\mathcal{G}_y(x)
= \frac{1}{1-x}\prod_{k=1}^{y}\frac{1}{1-x^k}. $$

More explicitly we have that
$$f(y,z) = [x^{yz}] \mathcal{F}_{y}(x) =[x^{yz}] \left(\frac{1}{1-x}\prod_{k=1}^{y}\frac{1}{1-x^k}\right)$$
where the operator $[x^{k}] $ is defined as follows:
$$\mathcal{A}(x) = a_0+a_1x+ ... +a_nx^n+ ... \implies [x^{k}]\mathcal{A}(x) = a_k.$$ For the asymptotics please consult Flajolet & Sedgewick's text on analytic combinatorics where you will find a wealth of information and techniques for extracting the asymptoics of the coefficients of $\mathcal{F}_{y}(x) $.