The distribution of product of normal matrices

Question

Assuming we have three matrices: $\mathbf{X} \in \mathbb{R}^{4096 \times 1024}$, $\mathbf{Q} \in \mathbb{R}^{1024 \times 2048}$, $\mathbf{K} \in \mathbb{R}^{2048 \times 1024}$. Where all elements of these matrices are independently sampled from the standard normal distribution $\mathcal{N}(0, 1)$.We compute the matrix $\mathbf{Y} = \mathbf{X} \mathbf{Q} \mathbf{K} \mathbf{X}^T \in \mathbb{R}^{4096 \times 4096}$ through matrix multiplication. The question is: What distribution do the individual elements of the matrix $\mathbf{Y}$ follow?How can we calculate its mean and variance?

Answer 1

The mean of the matrix elements of $Y$ is zero, because the matrix elements of $Q$ and $K$ average to zero. The variance is given by $$\mathbb{E}[Y_{ip}^2]=\sum_{jkl\alpha\beta\gamma}\mathbb{E}[X_{ij} Q_{jk} K_{kl} X_{pl} X_{i\alpha} Q_{\alpha\beta} K_{\beta\gamma} X_{p\gamma}]$$ $$=\sum_{jkl}\mathbb{E}[(X_{ij} Q_{jk} K_{kl} X_{pl})^2]=2048\sum_{j,l=1}^{1024}\mathbb{E}[(X_{ij}X_{pl})^2].$$ If $i\neq p$ this evaluates to $$\mathbb{E}[Y_{ip}^2]=2048\times(1024)^2=2147483648.$$

For $i=p$ instead $$\mathbb{E}[Y_{ii}^2]=2048\sum_{j,l=1}^{1024}\mathbb{E}[(X_{ij}X_{il})^2]=2048\left(\sum_{j,l=1,j\neq i}^{1024}\mathbb{E}[(X_{ij}X_{il})^2]+2048\sum_{j=1}^{1024}\mathbb{E}[X_{ij}^4]\right)$$ $$=2048\cdot 1024\cdot 1023+2048\cdot 1024\cdot 3=2151677952.$$

Concerning the full distribution: The paper The Product of Gaussian Matrices is Close to Gaussian by Li & Woodruff considers the problem where all $N\times N$ matrices in the product are independent; then an $n\times n$ submatrix of the matrix product is close in variation distance to a matrix with uncorrelated Gaussian elements if $n^2\ll N$. I expect this result also to apply to the case considered here, where one matrix is repeated in the product. In particular, the marginal distribution of a single matrix element of $Y$ will be close to a Gaussian.