How to prove the following inequality: Let X and Y be n*m matrices with real entries. Prove that det(XY^T)^2\leq det(XX``^T)det(YY^T)
If you replace determinants by traces, then this inequality is just CauchySchwartz for the inner product $(X,Y)=\mathop{\mathrm{tr}}(XY^T)$ on the space of matrices. Now, we just have to recall that determinants are traces: for a $n\times n$matrix $A$ we have $\det(A)=\mathop{\mathrm{tr}}(\Lambda^n(A))$, where $\Lambda^n(A)$ is the $n$th exterior power of the linear operator $A$. (Now that I took time to look at answers above I believe this is more or less precisely Wadim Zudilin's proof made a tiny bit more invariant.) 


If $n>m$ all the determinants are zero. The CauchyBinet formula allows you to write the determinant of $XY^t$ as the sum $$ \sum_{I\subset\lbrace1,2,\dots,m\rbrace}\det(X_IY_I^t) =\sum_{I\subset\lbrace1,2,\dots,m\rbrace}\det(X_I)\det(Y_I^t) $$ where the sum is over all $n$element subsets of $\lbrace1,2,\dots,m\rbrace$. Here $X_I$ and $Y_I$ are quadratic $n\times n$ matrices with columns of $X$ and $Y$ from the set $I$. The similar formulas are available for $\det(XX^t)$ and $\det(YY^t)$, so the wanted inequality reduces to verification of $$ \left(\sum_{I}x_Iy_I\right)^2\le \sum_{I}x_I^2\sum_Iy_I^2. $$ This a known inequality. 


Denotes vectorsrows of $X$ by $x_1$, $\dots$, $x_n$, of $Y$ by $y_1$, $\dots$, $y_n$, all $x_i$, $y_i$ belong to $\mathbb{R}^m$. Let $y_i=z_i+w_i$, where $z_i$ belongs to linear span of $x_1$,$\dots$,$x_n$, and $w_i$ is orthogonal to this linear span. If we replace $y_i$ to $z_i$, the left hand side does not change, while the right hand side may only decrease ($YY^t=ZZ^t+WW^t$ with natural notations, and all matrices are nonnegative definite, so all eigenvalues of $YY^t$ are at least the same as those of $ZZ^t$, hence $\det(YY^t)\geq \det(ZZ^t)$.) But if all $x_i$, $y_i$ belong to the same subspace of dimension $n$, then equality in inicial inequality occurs, as we may think that $m=n$. 

