Here is the proof for $N=2$. It contains also some, well, you may call it 'ideas', which theoretically may help in further cases. [Here][1] is a straightforward proof that for two-dimensional vectors $x=(x_1,x_2),y=(y_1,y_2)$ we have $$(x_1^2+x_2^2)(y_1^2+y_2^2)+(x_1y_1+x_2y_2)^2\geqslant \sqrt{(x_1^4+x_2^4)(y_1^4+y_2^4)}+x_1^2y_1^2+x_2^2y_2^2.$$ We use notations $\|x\|^2=\sum x_i^2$ for $x=(x_1,\dots,x_d)\in \mathbb R^d$, $(x,y)=\sum_{i=1}^d x_iy_i$ denotes the standard inner product. Next, I need an easy **Lemma.** For vectors $p,q,a$ in $\mathbb{R}^d$ we have $$2\frac{(p,a)\cdot (q,a)}{\|a\|^2}\leqslant \|p\|\cdot \|q\|+(p,q)$$ **Proof.** At first, replacing $a$ to its projection onto 2-plane containing $x$, $y$ increases LHS and does not change RHS. So we may suppose that $p,q,a$ lie in a 2-plane. Then if we denote angles between $p,a$ by $\alpha$ and $q,a$ by $\beta$ we have to prove $2\cos \alpha \cos \beta\leqslant 1+\cos(\alpha\pm \beta)$, this follows from identity $2\cos \alpha \cos \beta=\cos(\alpha+\beta)+\cos(\alpha-\beta)$. More involved **observation** is the following. Assume that $L$ is a subspace in $\mathbb{R}^d$ $T$ is a self-adjoint operator on $\mathbb{R}^d$. Then there exists a non-negative definite self-adjoint operator $S:L\rightarrow L$ such that $\|Su\|=\|Tu\|$ for any $u$ in $L$. **Proof.** Let $Q$ be any linear isometry from $T(L)$ to $L$ (if $T(L)$ has dimension less than that of $L$, $Q$ is defined on some extension of $T(L)$). Then $QT:L\rightarrow L$ and $\|QTu\|=\|Tu\|$ for $u\in L$. But $QT$ is not self-adjoint, well, take $S=\sqrt{(QT)^*QT}$. Now come to your question for $N=2$. Denote $x=X^{1/4} v_1$, $y=X^{1/4}v_2$, $T=X^{1/4}$, ${\rm tr}\, T^4=1$. Your inequality may be rewritten as $$ \left(\frac{\|x\|^4}{\|Tx\|^4}+\frac{(x,y)^2}{\|Tx\|^2 \|Ty\|^2}\right) \left(\frac{\|y\|^4}{\|Ty\|^4}+\frac{(x,y)^2}{\|Tx\|^2 \|Ty\|^2}\right)\geqslant \frac 4{({\rm tr}\, T^4)^2}, $$ where last denominator is added for making it homogeneous in $T$. We check it in two steps: at first, prove it for $S$ instead $T$, next, prove that ${\rm tr}\, S^4\leqslant {\rm tr}\, T^4$. The second step is a standard application of variational principle. If $\alpha\geqslant \beta\geqslant 0$ are eigenvalues of $S$, $\lambda_1\geqslant \dots \geqslant \lambda_n>0$ are eigenvalues of $T$, we see that there exists non-zero vector $u\in L$ such that $\lambda_1 \|u\|\geqslant\|Tu\|=\|Su\|=\alpha\cdot \|u\|$. It follows that $\alpha\leqslant \lambda_1$. Next, we choose a non-zero vector $u$ which lies both in $L$ and in a subspace $W$ of codimension 1 generated by all eigenvectors of $T$ corresponded to $\lambda_2,\lambda_3,\dots,\lambda_n$. Then $\lambda_2 \|u\|\geqslant \|Tu\|=\|Su\|\geqslant \beta\cdot \|u\|$, thus $\lambda_2\geqslant \beta$. The claim ${\rm tr}\, S^4\leqslant {\rm tr}\, T^4$ follows. Now the first step. By Cauchy-Bunyakovsky-Schwarz Inequality we have $$ \left(\frac{\|x\|^4}{\|Sx\|^4}+\frac{(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right) \left(\frac{\|y\|^4}{\|Sy\|^4}+\frac{(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right)\geqslant \left(\frac{\|x\|^2\cdot \|y\|^2+(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right)^2, $$ and it remains to prove that $$ \frac{\|x\|^2\cdot \|y\|^2+(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\geqslant \frac2{{\rm tr}\, S^4}. $$ We may suppose that $S$ is diagonal with diagonal elements $\alpha,\beta$. Also denote $x=(x_1,x_2),y=(y_1,y_2)$. For $a=(k_1^2,k_2^2),b=(x_1^2,x_2^2),p=(k_1^2,k_2^2)$ our lemma says that $$ 2\frac{\|Sx\|^2 \|Sy\|^2}{{\rm tr}\, S^4}\leqslant \sqrt{(x_1^4+x_2^4)(y_1^4+y_2^4)}+x_1^2y_1^2+x_2^2y_2^2, $$ and it remains to use out starting inequality. [1]: http://www.wolframalpha.com/input/?i=((x_1%5E2%2Bx_2%5E2)(y_1%5E2%2By_2%5E2)%2B(x_1*y_1%2Bx_2*y_2)%5E2-(x_1%5E2*y_1%5E2%2Bx_2%5E2*y_2%5E2))%5E2-(x_1%5E4%2Bx_2%5E4)(y_1%5E4%2By_2%5E4)