Let us denote the matrices in question by $\rho_{nm}$ and, first, consider the use of the measure generated by the Hilbert-Schmidt metric \begin{equation} \mbox{d} s^2_{HS}=\frac{1}{2} \mbox{Tr}[(\mbox{d} \rho_{nm})^2]. \end{equation}
Then, A. Lovas and A. Andai
LovasAndaiPaper MR3673324
have formally answered the question for the case of symmetric $\rho_{22}$, obtaining the value $\frac{29}{64}$ for the proportion ("separability probability") of this nine-dimensional set of "two-rebit density matrices" that remain positive-definite under the indicated operation of "partial transposition".
In their Conclusions, Lovas and Andai write: ``The structure of the unit ball in operator norm of $2\times 2$ matrices plays a critical role in separability probability of qubit-qubit and rebit-rebit quantum systems. It is quite surprising that the space of $2\times 2$ real or complex matrices seems simple, but to compute the volume of the set \begin{equation*} \Big\{\begin{pmatrix}a & b\\ c& e\end{pmatrix} \Big\vert\ a, b, c, e \in \mathbb{K}, \Big| \Big|{\begin{pmatrix} a & b\\ c& e\end{pmatrix}} \Big| \Big| <1,\ \ \Big| \Big|{\begin{pmatrix} a & \varepsilon b\\ \frac{c}{\varepsilon}& e \end{pmatrix}} <1 \Big| \Big|\Big\} \end{equation*} for a given parameter $\varepsilon\in [0,1]$, which is the value of the function $\chi_{d}(\varepsilon)$, is a very challenging problem. The gist of our considerations is that the behavior of the function $\chi_{d}(\varepsilon)$ determines the separability probabilities with respect to the Hilbert-Schmidt measure.'' (The operator norm $ \Big| \Big| \hspace{.15in} \Big| \Big|$ is the largest singular value or Schatten-$\infty$ norm.)
The function $\chi_{1}(\varepsilon)$--found employing an auxiliary "defect function"--which is used for the determination of the $\frac{29}{64}$ is given by
\begin{equation} \label{BasicFormula}
\tilde{\chi}_1 (\varepsilon ) = 1-\frac{4}{\pi^2}\int\limits_\varepsilon^1
\left(
s+\frac{1}{s}-
\frac{1}{2}\left(s-\frac{1}{s}\right)^2\log \left(\frac{1+s}{1-s}\right)
\right)\frac{1}{s}
\mbox{d} s
\end{equation}
\begin{equation}
= \frac{4}{\pi^2}\int\limits_0^\varepsilon
\left(
s+\frac{1}{s}-
\frac{1}{2}\left(s-\frac{1}{s}\right)^2\log \left(\frac{1+s}{1-s}\right)
\right)\frac{1}{s}
\mbox{d} s .
\end{equation}
Let us note that
$\tilde{\chi}_1 (\varepsilon )$ has a closed form,
\begin{equation} \label{poly}
\frac{2 \left(\varepsilon ^2 \left(4 \text{Li}_2(\varepsilon )-\text{Li}_2\left(\varepsilon
^2\right)\right)+\varepsilon ^4 \left(-\tanh ^{-1}(\varepsilon )\right)+\varepsilon ^3-\varepsilon
+\tanh ^{-1}(\varepsilon )\right)}{\pi ^2 \varepsilon ^2},
\end{equation}
where the polylogarithmic function is defined by the infinite sum
\begin{equation*}
\text{Li}_s (z) =
\sum\limits_{k=1}^\infty
\frac{z^k}{k^s},
\end{equation*}
for arbitrary complex $s$ and for all complex arguments $z$ with $|z|<1$.
Lovas and Andai left unanswered the (two-qubit) matter of (15-dimensional) Hermitian $\rho_{22}$.
In MasterLovasAndai MR3767844
Slater was able to construct--though yet without formalized proof--the much simpler \begin{equation} \label{BasicFormula2} \tilde{\chi}_2 (\varepsilon ) = \frac{1}{3} \varepsilon^2 (4-\varepsilon^2) \end{equation} leading to the two-qubit separability probability of $\frac{8}{33}$. (Also, in this paper, counterparts were given for quaternionic [$\tilde{\chi}_4 (\varepsilon ) = \frac{1}{35} \varepsilon^4 (84-64\varepsilon^2+15 \varepsilon^4)$ yielding $\frac{26}{323}$],...density matrices.)
However, for $n$ or $m$ greater than 2, no analogous formulas are yet available.
Extensive numerical (quasirandom estimation) investigations
have led to conjectures that for $n=3,m=2$ (or $n=2,m=3$) for symmetric ("rebit-retrit") density matrices the Hilbert-Schmidt probability in question is $\frac{860}{6561} =\frac{2 \cdot 5 \cdot 43}{3^8}$, and for Hermitian ("qubit-qutrit" density matrices, the corresponding probability is $\frac{27}{1000}=\frac{3^3}{2^3 \cdot 5^3}$.
However, despite these limited results pertaining to small $n,m$, Szarek, Bengtsson and Zyczkowski
StructureBody MR2200422 (2006i:81029)
were able to formally establish--specifically in the case of the Hilbert-Schmidt measure--that for all dimensions, both in the symmetric and Hermitian scenarios, the probability for the class of rank-$nm-1$ matrices is one-half that for the class of full rank ($nm$) matrices. The proof was accomplished by showing that the set of full rank ($nm$) matrices is "pyramid-decomposable", and hence is a body of constant height.
Ruskai and Werner
RuskaiWerner MR2525543 (2010h:81031)
have established that the probability in question is zero if the rank of the $n m \times n m$ density matrix is less than or equal to $\mbox{max}(n,m)$.
For rank-4 ($6 \times 6$) qubit-qutrit density matrices, certain numerical evidence suggests that the associated probability might be $\frac{1}{34}$ that of the rank-6 probability (conjectured, as indicated above, to be $\frac{27}{1000}$).
Additionally, other choices of measures on the density matrices have been considered (in particular, the "Bures", an example of an operator monotone measure)
[GeometryOfQuantumStates][4] MR3752196 (extensive review of first edition MR2230995 (2007k:81001))
for which a two-rebit estimate of 0.15709623 has been obtained, and a two-qubit conjecture of $\frac{25}{341} =\frac{5^2}{11 \cdot 31}$ advanced.
For asymptotic aspects of this question, see. Chap. 9 of
[AliceBobBanach][4] MR3699754