Let me preface this by saying that I don't know a reference - so if that's what you're really looking for, someone else will have to answer.
Let $k:=R/m$ denote the residue field (I'm assuming "commutative" was implicit in your question).
Lemma 1 : Suppose $X$ is a bounded below chain complex of finitely generated $R$-modules such that $X\otimes^L_R k = 0$. Then $X=0$.
This is a simple application of Nakayama's lemma and $H_0(X\otimes^L_R k) = H_0(X)\otimes_R k$ if $X$ is nonnegatively graded.
Proposition 1: Suppose $X$ is a bounded below chain complex of finitely generated $R$-modules. If $X\otimes^L_R k$ is perfect, then so is $X$.
"Perfect" here is what you call "finite homological dimension".
Proof : By induction on the number of nonzero homology groups of $X\otimes^L_Rk$.
The base case, where $X\otimes^L_Rk = 0$, is lemma 1.
Assuming $X$ is nonnegatively graded and the least nonzero homology group is $H_0$, let $k^n \to H_0(X\otimes_R^Lk)\cong H_0(X)\otimes_R k$ be an isomorphism, and lift it to a surjection (by Nakayama's lemma) $R^n\to H_0(X)$, which we can then lift to a morphism $R^n\to X$ in $D(R)$, and take its cofiber $Y$.
So we have a cofiber sequence $R^n\to X\to Y$. $Y$ is also equivalent to a bounded below chain complex of finitely generated $R$-modules and is perfect upon tensoring with $k$.
Furthermore if we tensor this cofiber sequence with $k$, the long exact sequence of homology groups shows that $Y\otimes^L_Rk$ has less nonzero homology groups that $X\otimes^L_Rk$ (it has the same homology groups except it has no $H_0$). By induction, it follows that $Y$ is perfect over $R$, and therefore so is $X$.
From there we can conclude about the non-straightforward direction in your question, namely:
Proposition 2: Suppose $M,N$ are bounded below complexes of finitely generated $R$-modules that aren't acyclic. If $M\otimes_R^L N$ is perfect, then so are $M,N$.
Proof: $-\otimes^L_Rk$ is symmetric monoidal and maps perfect $R$-complexes to perfect $k$-complexes, so by proposition 1 and lemma 1 it suffices to prove the result over $k$.
Over $k$ we have the Künneth formula and formality of chain complexes which make this into a straightforward result.
(note that lemma 1 is necessary here to ensure that $M\otimes_R^Lk$ and $N\otimes^L_Rk$ are nonzero)