Let us denote the matrices in question by $\rho_{nm}$ and, first, consider the use of the measure generated by the Hilbert-Schmidt metric
\begin{equation}
\mbox{d} s^2_{HS}=\frac{1}{2} \mbox{Tr}[(\mbox{d} \rho_{nm})^2].
\end{equation}
Then, A. Lovas and A. Andai
LovasAndaiPaper MR3673324
have formally answered the question for the case of symmetric $\rho_{22}$, obtaining the value $\frac{29}{64}$ for the proportion ("separability probability") of this nine-dimensional set of "two-rebit density matrices" that remain positive-definite under the indicated operation of "partial transposition".
In their Conclusions, Lovas and Andai write: ``The structure of the unit ball in operator norm of $2\times 2$ matrices plays a
critical role in separability probability of qubit-qubit and rebit-rebit
quantum systems.
It is quite surprising that the space of $2\times 2$ real or complex matrices
seems simple, but to compute the volume of the set
\begin{equation*}
\Big\{\begin{pmatrix}a & b\\ c& e\end{pmatrix} \Big\vert\ a, b, c, e \in \mathbb{K},
\Big| \Big|{\begin{pmatrix} a & b\\ c& e\end{pmatrix}} \Big| \Big| <1,\ \
\Big| \Big|{\begin{pmatrix} a & \varepsilon b\\ \frac{c}{\varepsilon}& e
\end{pmatrix}} <1 \Big| \Big|\Big\}
\end{equation*}
for a given parameter $\varepsilon\in [0,1]$, which is the value of
the function $\chi_{d}(\varepsilon)$, is a very challenging problem.
The gist of our considerations is that the behavior of the function
$\chi_{d}(\varepsilon)$ determines the separability probabilities with respect
to the Hilbert-Schmidt measure.'' (The operator norm $ \Big| \Big| \hspace{.15in} \Big| \Big|$ is the largest singular value or Schatten-$\infty$ norm.)
The function $\chi_{1}(\varepsilon)$--found employing an auxiliary "defect function"--which is used for the determination of the $\frac{29}{64}$ is given by
\begin{equation} \label{BasicFormula}
\tilde{\chi}_1 (\varepsilon ) = 1-\frac{4}{\pi^2}\int\limits_\varepsilon^1
\left(
s+\frac{1}{s}-
\frac{1}{2}\left(s-\frac{1}{s}\right)^2\log \left(\frac{1+s}{1-s}\right)
\right)\frac{1}{s}
\mbox{d} s
\end{equation}
\begin{equation}
= \frac{4}{\pi^2}\int\limits_0^\varepsilon
\left(
s+\frac{1}{s}-
\frac{1}{2}\left(s-\frac{1}{s}\right)^2\log \left(\frac{1+s}{1-s}\right)
\right)\frac{1}{s}
\mbox{d} s .
\end{equation}
Let us note that
$\tilde{\chi}_1 (\varepsilon )$ has a closed form,
\begin{equation} \label{poly}
\frac{2 \left(\varepsilon ^2 \left(4 \text{Li}_2(\varepsilon )-\text{Li}_2\left(\varepsilon
^2\right)\right)+\varepsilon ^4 \left(-\tanh ^{-1}(\varepsilon )\right)+\varepsilon ^3-\varepsilon
+\tanh ^{-1}(\varepsilon )\right)}{\pi ^2 \varepsilon ^2},
\end{equation}
where the polylogarithmic function is defined by the infinite sum
\begin{equation*}
\text{Li}_s (z) =
\sum\limits_{k=1}^\infty
\frac{z^k}{k^s},
\end{equation*}
for arbitrary complex $s$ and for all complex arguments $z$ with $|z|<1$.
Lovas and Andai left unanswered the (two-qubit) matter of (15-dimensional) Hermitian $\rho_{22}$.
In MasterLovasAndai MR3767844
Slater was able to construct--though yet without formalized proof--the much simpler
\begin{equation} \label{BasicFormula2}
\tilde{\chi}_2 (\varepsilon ) = \frac{1}{3} \varepsilon^2 (4-\varepsilon^2)
\end{equation}
leading to the two-qubit separability probability of $\frac{8}{33}$.
Also, in this paper, counterparts were given for quaternionic [$\tilde{\chi}_4 (\varepsilon ) = \frac{1}{35} \varepsilon^4 (84-64\varepsilon^2+15 \varepsilon^4)$ yielding $\frac{26}{323}$],...density matrices.
Then, these three formulas were incorporated into a "Master Lovas-Andai" formula--the index $d$ being a form of "Dyson-index" of random matrix theory-
\begin{equation}
\tilde{\chi}_d (\varepsilon ) = \frac{\varepsilon^d \Gamma(d+1)^3 \, _3\tilde{F}_2\left(-\frac{d}{2},\frac{d}{2},d;\frac{d}{2}+1,\frac{3 d}{2}+1;\varepsilon^2\right)}{\Gamma(\frac{d}{2}+1)^2},
\end{equation}
where the regularized hypergeometric function is indicated.
For $\alpha=\frac{d}{2}$, the desired probabilities ($\frac{29}{64}, \frac{8}{33},\ldots$) are yielded by
\begin{equation} \label{Hou1}
P(\alpha) =\Sigma_{i=0}^\infty f(\alpha+i),
\end{equation}
where
\begin{equation} \label{Hou2}
f(\alpha) = P(\alpha)-P(\alpha +1) = \frac{ q(\alpha) 2^{-4 \alpha -6} \Gamma{(3 \alpha +\frac{5}{2})} \Gamma{(5 \alpha +2})}{6 \Gamma{(\alpha +1)} \Gamma{(2 \alpha +3)}
\Gamma{(5 \alpha +\frac{13}{2})}},
\end{equation}
and
\begin{equation} \label{Hou3}
q(\alpha) = 185000 \alpha ^5+779750 \alpha ^4+1289125 \alpha ^3+1042015 \alpha ^2+410694 \alpha +63000 =
\end{equation}
\begin{equation}
\alpha \bigg(5 \alpha \Big(25 \alpha \big(2 \alpha (740 \alpha
+3119)+10313\big)+208403\Big)+410694\bigg)+63000.
\end{equation}
(Qing-Hu Hou helped in the derivation of this formula, using "Zeilberger's algorithm" ["creative telescoping"] ConciseFormula.)
An equivalent formula--now employing $d$ rather than $\alpha=\frac{d}{2}$--was given by C. F. Dunkl in App. D of MasterLovasAndai
\begin{equation}
\mathcal{P}(d) =3456^{d}\frac{\left( \frac{1}{2}\right) _{d/2}%
^{3}\left( \frac{7}{6}\right) _{d/2}^{2}\left( \frac{5}{6}\right)
_{d/2}^{2}\left( 2d\right) !}{\left( \frac{d}{2}\right) !\left( 3\right)
_{5d}}\sum_{i\geq0,j\geq0}^{i+j\leq d/2}\frac{\left( -\frac{d}{2}\right)
_{i+j}\left( \frac{d}{2}\right) _{j}\left( d\right) _{j}\left(
2+3d\right) _{i}\left( 1+d\right) _{i}}{\left( 2+\frac{5d}{2}\right)
_{i+j}\left( 1+\frac{d}{2}\right) _{j}i!j!\left( -2d\right) _{i}}.
\end{equation}
However, for dimensions $n$ or $m$ greater than 2, no analogous formulas are yet available.
Extensive numerical (quasirandom estimation) investigations
NumericalExact
have led to conjectures that for $n=3,m=2$ (or $n=2,m=3$) for symmetric ("rebit-retrit") density matrices the Hilbert-Schmidt probability in question is $\frac{860}{6561} =\frac{2 \cdot 5 \cdot 43}{3^8}$, and for Hermitian ("qubit-qutrit" density matrices, the corresponding probability is $\frac{27}{1000}=\frac{3^3}{2^3 \cdot 5^3}$.
However, despite these limited results pertaining to small $n,m$, Szarek, Bengtsson and Zyczkowski
StructureBody MR2200422 (2006i:81029)
were able to formally establish--specifically in the case of the Hilbert-Schmidt measure--that for all dimensions, both in the symmetric and Hermitian scenarios, the probability for the class of rank-$nm-1$ matrices is one-half that for the class of full rank ($nm$) matrices. The proof was accomplished by showing that the set of full rank ($nm$) matrices is "pyramid-decomposable", and hence is a body of constant height.
Ruskai and Werner
RuskaiWerner MR2525543 (2010h:81031)
have established that the probability in question is zero if the rank of the $n m \times n m$ density matrix is less than or equal to $\mbox{max}(n,m)$.
For rank-4 ($6 \times 6$) qubit-qutrit density matrices, certain numerical evidence suggests that the associated probability might be $\frac{1}{34}$ that of the rank-6 probability (conjectured, as indicated above, to be $\frac{27}{1000}$).
Additionally, other choices of measures on the density matrices have been considered (in particular, the "Bures", an example of an operator monotone measure)
[GeometryOfQuantumStates][4] MR3752196 (extensive review of first edition MR2230995 (2007k:81001))
for which a two-rebit estimate of 0.15709623 has been obtained, and a two-qubit conjecture of $\frac{25}{341} =\frac{5^2}{11 \cdot 31}$ advanced.
For asymptotic aspects of this question, see. Chap. 9 of
[AliceBobBanach][4] MR3699754