Here is the proof for $N=2$. It contains also some, well, you may call it 'ideas', which theoretically may help in further cases. Here is a straightforward proof that for two-dimensional vectors $x=(x_1,x_2),y=(y_1,y_2)$ we have $$(x_1^2+x_2^2)(y_1^2+y_2^2)+(x_1y_1+x_2y_2)^2\geqslant \sqrt{(x_1^4+x_2^4)(y_1^4+y_2^4)}+x_1^2y_1^2+x_2^2y_2^2.$$ We use notations $\|x\|^2=\sum x_i^2$ for $x=(x_1,\dots,x_d)\in \mathbb R^d$, $(x,y)=\sum_{i=1}^d x_iy_i$ denotes the standard inner product. Next, I need an easy

Lemma. For vectors $p,q,a$ in $\mathbb{R}^d$ we have $$2\frac{(p,a)\cdot (q,a)}{\|a\|^2}\leqslant \|p\|\cdot \|q\|+(p,q)$$

Proof. At first, replacing $a$ to its projection onto 2-plane containing $x$, $y$ increases LHS and does not change RHS. So we may suppose that $p,q,a$ lie in a 2-plane. Then if we denote angles between $p,a$ by $\alpha$ and $q,a$ by $\beta$ we have to prove $2\cos \alpha \cos \beta\leqslant 1+\cos(\alpha\pm \beta)$, this follows from identity $2\cos \alpha \cos \beta=\cos(\alpha+\beta)+\cos(\alpha-\beta)$.

More involved observation is the following. Assume that $L$ is a subspace in $\mathbb{R}^d$ $T$ is a self-adjoint operator on $\mathbb{R}^d$. Then there exists a non-negative definite self-adjoint operator $S:L\rightarrow L$ such that $\|Su\|=\|Tu\|$ for any $u$ in $L$.

Proof. Let $Q$ be any linear isometry from $T(L)$ to $L$ (if $T(L)$ has dimension less than that of $L$, $Q$ is defined on some extension of $T(L)$). Then $QT:L\rightarrow L$ and $\|QTu\|=\|Tu\|$ for $u\in L$. But $QT$ is not self-adjoint, well, take $S=\sqrt{(QT)^*QT}$.

Now come to your question for $N=2$. Denote $x=X^{1/4} v_1$, $y=X^{1/4}v_2$, $T=X^{1/4}$, ${\rm tr}\, T^4=1$. Your inequality may be rewritten as $$ \left(\frac{\|x\|^4}{\|Tx\|^4}+\frac{(x,y)^2}{\|Tx\|^2 \|Ty\|^2}\right) \left(\frac{\|y\|^4}{\|Ty\|^4}+\frac{(x,y)^2}{\|Tx\|^2 \|Ty\|^2}\right)\geqslant \frac 4{({\rm tr}\, T^4)^2}, $$ where last denominator is added for making it homogeneous in $T$. We check it in two steps: at first, prove it for $S$ instead $T$, next, prove that ${\rm tr}\, S^4\leqslant {\rm tr}\, T^4$. The second step is a standard application of variational principle. If $\alpha\geqslant \beta$\geqslant 0 are eigenvalues of $S$, $\lambda_1\geqslant \dots \geqslant \lambda_n>0$ are eigenvalues of $T$, we see that there exists non-zero vector $u\in L$ such that $\lambda_1 \|u\|\geqslant\|Tu\|=\|Su\|=\alpha\cdot \|u\|$. It follows that $\alpha\leqslant \lambda_1$. Next, we choose a non-zero vector $u$ which lies both in $L$ and in a subspace $W$ of codimension 1 generated by all eigenvectors of $T$ corresponded to $\lambda_2,\lambda_3,\dots,\lambda_n$. Then $\lambda_2 \|u\|\geqslant \|Tu\|=\|Su\|\geqslant \beta\cdot \|u\|$, thus $\lambda_2\geqslant \beta$. The claim ${\rm tr}\, S^4\leqslant {\rm tr}\, T^4$ follows.

Now the first step. By Cauchy-Bunyakovsky-Schwarz Inequality we have $$ \left(\frac{\|x\|^4}{\|Sx\|^4}+\frac{(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right) \left(\frac{\|y\|^4}{\|Sy\|^4}+\frac{(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right)\geqslant \left(\frac{\|x\|^2\cdot \|y\|^2+(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right)^2, $$ and it remains to prove that $$ \frac{\|x\|^2\cdot \|y\|^2+(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\geqslant \frac2{{\rm tr}\, S^4}. $$ We may suppose that $S$ is diagonal with diagonal elements $\alpha,\beta$. Also denote $x=(x_1,x_2),y=(y_1,y_2)$. For $a=(k_1^2,k_2^2),b=(x_1^2,x_2^2),p=(k_1^2,k_2^2)$ our lemma says that $$ 2\frac{\|Sx\|^2 \|Sy\|^2}{{\rm tr}\, S^4}\leqslant \sqrt{(x_1^4+x_2^4)(y_1^4+y_2^4)}+x_1^2y_1^2+x_2^2y_2^2, $$ and it remains to use out starting inequality.

Here is the proof for $N=2$. It contains also some, well, you may call it 'ideas', which theoretically may help in further cases. Here is a straightforward proof that for two-dimensional vectors $x=(x_1,x_2),y=(y_1,y_2)$ we have $$(x_1^2+x_2^2)(y_1^2+y_2^2)+(x_1y_1+x_2y_2)^2\geqslant \sqrt{(x_1^4+x_2^4)(y_1^4+y_2^4)}+x_1^2y_1^2+x_2^2y_2^2.$$ We use notations $\|x\|^2=\sum x_i^2$ for $x=(x_1,\dots,x_d)\in \mathbb R^d$, $(x,y)=\sum_{i=1}^d x_iy_i$ denotes the standard inner product. Next, I need an easy

Lemma. For vectors $p,q,a$ in $\mathbb{R}^d$ we have $$2\frac{(p,a)\cdot (q,a)}{\|a\|^2}\leqslant \|p\|\cdot \|q\|+(p,q)$$

Proof. At first, replacing $a$ to its projection onto 2-plane containing $x$, $y$ increases LHS and does not change RHS. So we may suppose that $p,q,a$ lie in a 2-plane. Then if we denote angles between $p,a$ by $\alpha$ and $q,a$ by $\beta$ we have to prove $2\cos \alpha \cos \beta\leqslant 1+\cos(\alpha\pm \beta)$, this follows from identity $2\cos \alpha \cos \beta=\cos(\alpha+\beta)+\cos(\alpha-\beta)$.

More involved observation is the following. Assume that $L$ is a subspace in $\mathbb{R}^d$ $T$ is a self-adjoint operator on $\mathbb{R}^d$. Then there exists a non-negative definite self-adjoint operator $S:L\rightarrow L$ such that $\|Su\|=\|Tu\|$ for any $u$ in $L$.

Proof. Let $Q$ be any linear isometry from $T(L)$ to $L$ (if $T(L)$ has dimension less than that of $L$, $Q$ is defined on some extension of $T(L)$). Then $QT:L\rightarrow L$ and $\|QTu\|=\|Tu\|$ for $u\in L$. But $QT$ is not self-adjoint, well, take $S=\sqrt{(QT)^*QT}$.

Now come to your question for $N=2$. Denote $x=X^{1/4} v_1$, $y=X^{1/4}v_2$, $T=X^{1/4}$, ${\rm tr}\, T^4=1$. Your inequality may be rewritten as $$ \left(\frac{\|x\|^4}{\|Tx\|^4}+\frac{(x,y)^2}{\|Tx\|^2 \|Ty\|^2}\right) \left(\frac{\|y\|^4}{\|Ty\|^4}+\frac{(x,y)^2}{\|Tx\|^2 \|Ty\|^2}\right)\geqslant \frac 4{({\rm tr}\, T^4)^2}, $$ where last denominator is added for making it homogeneous in $T$. We check it in two steps: at first, prove it for $S$ instead $T$, next, prove that ${\rm tr}\, S^4\leqslant {\rm tr}\, T^4$. The second step is a standard application of variational principle. If $\alpha\geqslant \beta\geqslant 0$ are eigenvalues of $S$, $\lambda_1\geqslant \dots \geqslant \lambda_n>0$ are eigenvalues of $T$, we see that there exists non-zero vector $u\in L$ such that $\lambda_1 \|u\|\geqslant\|Tu\|=\|Su\|=\alpha\cdot \|u\|$. It follows that $\alpha\leqslant \lambda_1$. Next, we choose a non-zero vector $u$ which lies both in $L$ and in a subspace $W$ of codimension 1 generated by all eigenvectors of $T$ corresponded to $\lambda_2,\lambda_3,\dots,\lambda_n$. Then $\lambda_2 \|u\|\geqslant \|Tu\|=\|Su\|\geqslant \beta\cdot \|u\|$, thus $\lambda_2\geqslant \beta$. The claim ${\rm tr}\, S^4\leqslant {\rm tr}\, T^4$ follows.

Now the first step. By Cauchy-Bunyakovsky-Schwarz Inequality we have $$ \left(\frac{\|x\|^4}{\|Sx\|^4}+\frac{(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right) \left(\frac{\|y\|^4}{\|Sy\|^4}+\frac{(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right)\geqslant \left(\frac{\|x\|^2\cdot \|y\|^2+(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right)^2, $$ and it remains to prove that $$ \frac{\|x\|^2\cdot \|y\|^2+(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\geqslant \frac2{{\rm tr}\, S^4}. $$ We may suppose that $S$ is diagonal with diagonal elements $\alpha,\beta$. Also denote $x=(x_1,x_2),y=(y_1,y_2)$. For $a=(k_1^2,k_2^2),b=(x_1^2,x_2^2),p=(k_1^2,k_2^2)$ our lemma says that $$ 2\frac{\|Sx\|^2 \|Sy\|^2}{{\rm tr}\, S^4}\leqslant \sqrt{(x_1^4+x_2^4)(y_1^4+y_2^4)}+x_1^2y_1^2+x_2^2y_2^2, $$ and it remains to use out starting inequality.

Here is the proof for $N=2$. It contains also some, well, you may call it 'ideas', which theoretically may help in further cases. Here is a straightforward proof that for two-dimensional vectors $x=(x_1,x_2),y=(y_1,y_2)$ we have $$(x_1^2+x_2^2)(y_1^2+y_2^2)+(x_1y_1+x_2y_2)^2\geqslant \sqrt{(x_1^4+x_2^4)(y_1^4+y_2^4)}+x_1^2y_1^2+x_2^2y_2^2.$$ We use notations $\|x\|^2=\sum x_i^2$ for $x=(x_1,\dots,x_d)\in \mathbb R^d$, $(x,y)=\sum_{i=1}^d x_iy_i$ denotes the standard inner product. Next, I need an easy

Lemma. For vectors $p,q,a$ in $\mathbb{R}^d$ we have $$2\frac{(p,a)\cdot (q,a)}{\|a\|^2}\leqslant \|p\|\cdot \|q\|+(p,q)$$

Proof. At first, replacing $a$ to its projection onto 2-plane containing $x$, $y$ increases LHS and does not change RHS. So we may suppose that $p,q,a$ lie in a 2-plane. Then if we denote angles between $p,a$ by $\alpha$ and $q,a$ by $\beta$ we have to prove $2\cos \alpha \cos \beta\leqslant 1+\cos(\alpha\pm \beta)$, this follows from identity $2\cos \alpha \cos \beta=\cos(\alpha+\beta)+\cos(\alpha-\beta)$.

More involved observation is the following. Assume that $x,y\in \mathbb{R}^d$ are linearly independent vectors and $T$ is a self-adjoint operator on $\mathbb{R}^d$. Denote by $L$ a 2-plane generated by $x$, $y$. Then there exists another self-adjoint operator $S:L\rightarrow L$ such that $\|Su\|=\|Tu\|$ for any $u$ in $L$.

Proof. Choose some points $A$, $B$ in $L$ so that triangle $0AB$ is congruent to $0(Tx)(Ty)$. If we rotate triangle $0AB$ centered in 0, then $(A,y)-(x,B)$ is a continuous function which changes its sign after rotation onto $\pi$. Hence it achieves zero value. So, we may suppose that $(A,y)=(x,B)$. Take $Sx=A$, $Sy=B$.

Now come to your question for $N=2$. Denote $x=X^{1/4} v_1$, $y=X^{1/4}v_2$, $T=X^{1/4}$, ${\rm tr}\, T^4=1$. Your inequality may be rewritten as $$ \left(\frac{\|x\|^4}{\|Tx\|^4}+\frac{(x,y)^2}{\|Tx\|^2 \|Ty\|^2}\right) \left(\frac{\|y\|^4}{\|Ty\|^4}+\frac{(x,y)^2}{\|Tx\|^2 \|Ty\|^2}\right)\geqslant \frac 4{({\rm tr}\, T^4)^2}, $$ where last denominator is added for making it homogeneous in $T$. We check it in two steps: at first, prove it for $S$ instead $T$, next, prove that ${\rm tr}\, S^4\leqslant {\rm tr}\, T^4$. The second step is a standard application of variational principle. If $\alpha,\beta$ are eigenvalues of $S$, $|\alpha|\geqslant |\beta|\geqslant 0$, $\lambda_1\geqslant \dots \geqslant \lambda_n>0$ are eigenvalues of $T$, we see that there exists non-zero vector $u\in L$ such that $\lambda_1 \|u\|\geqslant\|Tu\|=\|Su\|=|\alpha|\cdot \|u\|$. It follows that $|\alpha|\leqslant \lambda_1$. Next, we choose a non-zero vector $u$ which lies both in $L$ and in a subspace $W$ of codimension 1 generated by all eigenvectors of $T$ corresponded to $\lambda_2,\lambda_3,\dots,\lambda_n$. Then $\lambda_2 \|u\|\geqslant \|Tu\|=\|Su\|\geqslant |\beta|\cdot \|u\|$, thus $\lambda_2\geqslant |\beta|$. The claim ${\rm tr}\, S^4\leqslant {\rm tr}\, T^4$ follows.

Now the first step. By Cauchy-Bunyakovsky-Schwarz Inequality we have $$ \left(\frac{\|x\|^4}{\|Sx\|^4}+\frac{(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right) \left(\frac{\|y\|^4}{\|Sy\|^4}+\frac{(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right)\geqslant \left(\frac{\|x\|^2\cdot \|y\|^2+(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right)^2, $$ and it remains to prove that $$ \frac{\|x\|^2\cdot \|y\|^2+(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\geqslant \frac2{{\rm tr}\, S^4}. $$ We may suppose that $S$ is diagonal with diagonal elements $\alpha,\beta$. Also denote $x=(x_1,x_2),y=(y_1,y_2)$. For $a=(k_1^2,k_2^2),b=(x_1^2,x_2^2),p=(k_1^2,k_2^2)$ our lemma says that $$ 2\frac{\|Sx\|^2 \|Sy\|^2}{{\rm tr}\, S^4}\leqslant \sqrt{(x_1^4+x_2^4)(y_1^4+y_2^4)}+x_1^2y_1^2+x_2^2y_2^2, $$ and it remains to use out starting inequality.

Here is the proof for $N=2$. It contains also some, well, you may call it 'ideas', which theoretically may help in further cases. Here is a straightforward proof that for two-dimensional vectors $x=(x_1,x_2),y=(y_1,y_2)$ we have $$(x_1^2+x_2^2)(y_1^2+y_2^2)+(x_1y_1+x_2y_2)^2\geqslant \sqrt{(x_1^4+x_2^4)(y_1^4+y_2^4)}+x_1^2y_1^2+x_2^2y_2^2.$$ We use notations $\|x\|^2=\sum x_i^2$ for $x=(x_1,\dots,x_d)\in \mathbb R^d$, $(x,y)=\sum_{i=1}^d x_iy_i$ denotes the standard inner product. Next, I need an easy

Lemma. For vectors $p,q,a$ in $\mathbb{R}^d$ we have $$2\frac{(p,a)\cdot (q,a)}{\|a\|^2}\leqslant \|p\|\cdot \|q\|+(p,q)$$

Proof. At first, replacing $a$ to its projection onto 2-plane containing $x$, $y$ increases LHS and does not change RHS. So we may suppose that $p,q,a$ lie in a 2-plane. Then if we denote angles between $p,a$ by $\alpha$ and $q,a$ by $\beta$ we have to prove $2\cos \alpha \cos \beta\leqslant 1+\cos(\alpha\pm \beta)$, this follows from identity $2\cos \alpha \cos \beta=\cos(\alpha+\beta)+\cos(\alpha-\beta)$.

More involved observation is the following. Assume that $L$ is a subspace in $\mathbb{R}^d$ $T$ is a self-adjoint operator on $\mathbb{R}^d$. Then there exists a non-negative definite self-adjoint operator $S:L\rightarrow L$ such that $\|Su\|=\|Tu\|$ for any $u$ in $L$.

Proof. Let $Q$ be any linear isometry from $T(L)$ to $L$ (if $T(L)$ has dimension less than that of $L$, $Q$ is defined on some extension of $T(L)$). Then $QT:L\rightarrow L$ and $\|QTu\|=\|Tu\|$ for $u\in L$. But $QT$ is not self-adjoint, well, take $S=\sqrt{(QT)^*QT}$.

Now come to your question for $N=2$. Denote $x=X^{1/4} v_1$, $y=X^{1/4}v_2$, $T=X^{1/4}$, ${\rm tr}\, T^4=1$. Your inequality may be rewritten as $$ \left(\frac{\|x\|^4}{\|Tx\|^4}+\frac{(x,y)^2}{\|Tx\|^2 \|Ty\|^2}\right) \left(\frac{\|y\|^4}{\|Ty\|^4}+\frac{(x,y)^2}{\|Tx\|^2 \|Ty\|^2}\right)\geqslant \frac 4{({\rm tr}\, T^4)^2}, $$ where last denominator is added for making it homogeneous in $T$. We check it in two steps: at first, prove it for $S$ instead $T$, next, prove that ${\rm tr}\, S^4\leqslant {\rm tr}\, T^4$. The second step is a standard application of variational principle. If $\alpha\geqslant \beta$\geqslant 0 are eigenvalues of $S$, $\lambda_1\geqslant \dots \geqslant \lambda_n>0$ are eigenvalues of $T$, we see that there exists non-zero vector $u\in L$ such that $\lambda_1 \|u\|\geqslant\|Tu\|=\|Su\|=\alpha\cdot \|u\|$. It follows that $\alpha\leqslant \lambda_1$. Next, we choose a non-zero vector $u$ which lies both in $L$ and in a subspace $W$ of codimension 1 generated by all eigenvectors of $T$ corresponded to $\lambda_2,\lambda_3,\dots,\lambda_n$. Then $\lambda_2 \|u\|\geqslant \|Tu\|=\|Su\|\geqslant \beta\cdot \|u\|$, thus $\lambda_2\geqslant \beta$. The claim ${\rm tr}\, S^4\leqslant {\rm tr}\, T^4$ follows.

Now the first step. By Cauchy-Bunyakovsky-Schwarz Inequality we have $$ \left(\frac{\|x\|^4}{\|Sx\|^4}+\frac{(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right) \left(\frac{\|y\|^4}{\|Sy\|^4}+\frac{(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right)\geqslant \left(\frac{\|x\|^2\cdot \|y\|^2+(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\right)^2, $$ and it remains to prove that $$ \frac{\|x\|^2\cdot \|y\|^2+(x,y)^2}{\|Sx\|^2 \|Sy\|^2}\geqslant \frac2{{\rm tr}\, S^4}. $$ We may suppose that $S$ is diagonal with diagonal elements $\alpha,\beta$. Also denote $x=(x_1,x_2),y=(y_1,y_2)$. For $a=(k_1^2,k_2^2),b=(x_1^2,x_2^2),p=(k_1^2,k_2^2)$ our lemma says that $$ 2\frac{\|Sx\|^2 \|Sy\|^2}{{\rm tr}\, S^4}\leqslant \sqrt{(x_1^4+x_2^4)(y_1^4+y_2^4)}+x_1^2y_1^2+x_2^2y_2^2, $$ and it remains to use out starting inequality.