Skip to main content
Shortened the title
Source Link

Eigenvalue perturbation of a symmetric matrix by the outer product of manya random orthonormal columnsorthogonal projection

Given fixed real symmetric $D\in\mathbb{R}^{n\times n}$ with $n$ distinct eigenvalues, let $U$ be a random orthogonal matrix selected uniformly from the space of $n\times n$ orthogonal matrices, and denote the first $m$ columns of $U$ as the rectangular matrix $Q\in\mathbb{R}^{n\times m}$. Then we say that the matrix $QQ^T$ is a random orthogonal projection over $\mathbb{R}^n$ of rank $m$.

We are interested in the quantity $$\mu=\lambda_\min(D+QQ^T) - \lambda_\min(D)$$ for many columns, i.e. withwhen the rank of the projection satisfies $m\in\Omega(n)$. Of course, $0 \le \mu \le 1$. But given that $Q$ is randomly selected, the decoherence between the bases of $D$ and $Q$ causes the distribution of $\mu$ to lie strictly in between the two extremes. For sufficiently large $n$ the following statements should hold:

  1. The event $\mu=0$ occurs with probability zero.
  2. The expectation $\mathbb{E}\{\mu\}$ is bounded from below by an absolute constant.
  3. The distribution of $\mu$ concentrates about $\mathbb{E}\{\mu\}$.

Indeed, all three statements are readily confirmed using numerical simulations. But how might we go about proving these statements?


Numerical example. We sample $\mu$ for $n\in\{10,30,100,300\}$ and $m=n/4$, performing 1000 trials for each $n$. Initially, $D$ is set to be a diagonal matrix with random Gaussian entries, and this is fixed for all trials.

Numerical example


Remark 1. Since $D$ is fixed, we can, WLOG, assume that $D$ is positive definite and / or diagonal.

Remark 2. Many of the existing work on random orthogonal projections use standard Gaussians to approximate a few orthonormal columns. But with as many columns as $m\in\Omega(n)$, the approach is no longer valid. See Jiang, Tiefeng. "How many entries of a typical orthogonal matrix can be approximated by independent normals?" The Annals of Probability 34.4 (2006): 1497-1529.

Eigenvalue perturbation of a symmetric matrix by the outer product of many random orthonormal columns

Given fixed real symmetric $D\in\mathbb{R}^{n\times n}$ with $n$ distinct eigenvalues, let $U$ be a random orthogonal matrix selected uniformly from the space of $n\times n$ orthogonal matrices, and denote the first $m$ columns of $U$ as the rectangular matrix $Q\in\mathbb{R}^{n\times m}$.

We are interested in the quantity $$\mu=\lambda_\min(D+QQ^T) - \lambda_\min(D)$$ for many columns, i.e. with $m\in\Omega(n)$. Of course, $0 \le \mu \le 1$. But given that $Q$ is randomly selected, the decoherence between the bases of $D$ and $Q$ causes the distribution of $\mu$ to lie strictly in between the two extremes. For sufficiently large $n$ the following statements should hold:

  1. The event $\mu=0$ occurs with probability zero.
  2. The expectation $\mathbb{E}\{\mu\}$ is bounded from below by an absolute constant.
  3. The distribution of $\mu$ concentrates about $\mathbb{E}\{\mu\}$.

Indeed, all three statements are readily confirmed using numerical simulations. But how might we go about proving these statements?


Numerical example. We sample $\mu$ for $n\in\{10,30,100,300\}$ and $m=n/4$, performing 1000 trials for each $n$. Initially, $D$ is set to be a diagonal matrix with random Gaussian entries, and this is fixed for all trials.

Numerical example


Remark 1. Since $D$ is fixed, we can, WLOG, assume that $D$ is positive definite and / or diagonal.

Remark 2. Many of the existing work on random orthogonal projections use standard Gaussians to approximate a few orthonormal columns. But with as many columns as $m\in\Omega(n)$, the approach is no longer valid. See Jiang, Tiefeng. "How many entries of a typical orthogonal matrix can be approximated by independent normals?" The Annals of Probability 34.4 (2006): 1497-1529.

Eigenvalue perturbation of a symmetric matrix by a random orthogonal projection

Given fixed real symmetric $D\in\mathbb{R}^{n\times n}$ with $n$ distinct eigenvalues, let $U$ be a random orthogonal matrix selected uniformly from the space of $n\times n$ orthogonal matrices, and denote the first $m$ columns of $U$ as the rectangular matrix $Q\in\mathbb{R}^{n\times m}$. Then we say that the matrix $QQ^T$ is a random orthogonal projection over $\mathbb{R}^n$ of rank $m$.

We are interested in the quantity $$\mu=\lambda_\min(D+QQ^T) - \lambda_\min(D)$$ when the rank of the projection satisfies $m\in\Omega(n)$. Of course, $0 \le \mu \le 1$. But given that $Q$ is randomly selected, the decoherence between the bases of $D$ and $Q$ causes the distribution of $\mu$ to lie strictly in between the two extremes. For sufficiently large $n$ the following statements should hold:

  1. The event $\mu=0$ occurs with probability zero.
  2. The expectation $\mathbb{E}\{\mu\}$ is bounded from below by an absolute constant.
  3. The distribution of $\mu$ concentrates about $\mathbb{E}\{\mu\}$.

Indeed, all three statements are readily confirmed using numerical simulations. But how might we go about proving these statements?


Numerical example. We sample $\mu$ for $n\in\{10,30,100,300\}$ and $m=n/4$, performing 1000 trials for each $n$. Initially, $D$ is set to be a diagonal matrix with random Gaussian entries, and this is fixed for all trials.

Numerical example


Remark 1. Since $D$ is fixed, we can, WLOG, assume that $D$ is positive definite and / or diagonal.

Remark 2. Many of the existing work on random orthogonal projections use standard Gaussians to approximate a few orthonormal columns. But with as many columns as $m\in\Omega(n)$, the approach is no longer valid. See Jiang, Tiefeng. "How many entries of a typical orthogonal matrix can be approximated by independent normals?" The Annals of Probability 34.4 (2006): 1497-1529.

mistake in description of numerical experiment
Source Link

Given fixed real symmetric $D\in\mathbb{R}^{n\times n}$ with $n$ distinct eigenvalues, let $U$ be a random orthogonal matrix selected uniformly from the space of $n\times n$ orthogonal matrices, and denote the first $m$ columns of $U$ as the rectangular matrix $Q\in\mathbb{R}^{n\times m}$.

We are interested in the quantity $$\mu=\lambda_\min(D+QQ^T) - \lambda_\min(D)$$ for many columns, i.e. with $m\in\Omega(n)$. Of course, $0 \le \mu \le 1$. But given that $Q$ is randomly selected, the decoherence between the bases of $D$ and $Q$ causes the distribution of $\mu$ to lie strictly in between the two extremes. For sufficiently large $n$ the following statements should hold:

  1. The event $\mu=0$ occurs with probability zero.
  2. The expectation $\mathbb{E}\{\mu\}$ is bounded from below by an absolute constant.
  3. The distribution of $\mu$ concentrates about $\mathbb{E}\{\mu\}$.

Indeed, all three statements are readily confirmed using numerical simulations. But how might we go about proving these statements?


Numerical example. We sample $\mu$ for $n\in\{10,30,100,300\}$ and $m=n/4$, performing 1000 trials for each $n$. In each trialInitially, $D$ is set to be a diagonal matrix with random Gaussian entries, and this is fixed for all trials.   

Numerical example


Remark 1. Since $D$ is fixed, we can, WLOG, assume that $D$ is positive definite and / or diagonal.

Remark 2. Many of the existing work on random orthogonal projections use standard Gaussians to approximate a few orthonormal columns. But with as many columns as $m\in\Omega(n)$, the approach is no longer valid. See Jiang, Tiefeng. "How many entries of a typical orthogonal matrix can be approximated by independent normals?" The Annals of Probability 34.4 (2006): 1497-1529.

Given fixed real symmetric $D\in\mathbb{R}^{n\times n}$ with $n$ distinct eigenvalues, let $U$ be a random orthogonal matrix selected uniformly from the space of $n\times n$ orthogonal matrices, and denote the first $m$ columns of $U$ as the rectangular matrix $Q\in\mathbb{R}^{n\times m}$.

We are interested in the quantity $$\mu=\lambda_\min(D+QQ^T) - \lambda_\min(D)$$ for many columns, i.e. with $m\in\Omega(n)$. Of course, $0 \le \mu \le 1$. But given that $Q$ is randomly selected, the decoherence between the bases of $D$ and $Q$ causes the distribution of $\mu$ to lie strictly in between the two extremes. For sufficiently large $n$ the following statements should hold:

  1. The event $\mu=0$ occurs with probability zero.
  2. The expectation $\mathbb{E}\{\mu\}$ is bounded from below by an absolute constant.
  3. The distribution of $\mu$ concentrates about $\mathbb{E}\{\mu\}$.

Indeed, all three statements are readily confirmed using numerical simulations. But how might we go about proving these statements?


Numerical example. We sample $\mu$ for $n\in\{10,30,100,300\}$ and $m=n/4$, performing 1000 trials for each $n$. In each trial, $D$ is set to be a diagonal matrix with random Gaussian entries.  Numerical example


Remark 1. Since $D$ is fixed, we can, WLOG, assume that $D$ is positive definite and / or diagonal.

Remark 2. Many of the existing work on random orthogonal projections use standard Gaussians to approximate a few orthonormal columns. But with as many columns as $m\in\Omega(n)$, the approach is no longer valid. See Jiang, Tiefeng. "How many entries of a typical orthogonal matrix can be approximated by independent normals?" The Annals of Probability 34.4 (2006): 1497-1529.

Given fixed real symmetric $D\in\mathbb{R}^{n\times n}$ with $n$ distinct eigenvalues, let $U$ be a random orthogonal matrix selected uniformly from the space of $n\times n$ orthogonal matrices, and denote the first $m$ columns of $U$ as the rectangular matrix $Q\in\mathbb{R}^{n\times m}$.

We are interested in the quantity $$\mu=\lambda_\min(D+QQ^T) - \lambda_\min(D)$$ for many columns, i.e. with $m\in\Omega(n)$. Of course, $0 \le \mu \le 1$. But given that $Q$ is randomly selected, the decoherence between the bases of $D$ and $Q$ causes the distribution of $\mu$ to lie strictly in between the two extremes. For sufficiently large $n$ the following statements should hold:

  1. The event $\mu=0$ occurs with probability zero.
  2. The expectation $\mathbb{E}\{\mu\}$ is bounded from below by an absolute constant.
  3. The distribution of $\mu$ concentrates about $\mathbb{E}\{\mu\}$.

Indeed, all three statements are readily confirmed using numerical simulations. But how might we go about proving these statements?


Numerical example. We sample $\mu$ for $n\in\{10,30,100,300\}$ and $m=n/4$, performing 1000 trials for each $n$. Initially, $D$ is set to be a diagonal matrix with random Gaussian entries, and this is fixed for all trials. 

Numerical example


Remark 1. Since $D$ is fixed, we can, WLOG, assume that $D$ is positive definite and / or diagonal.

Remark 2. Many of the existing work on random orthogonal projections use standard Gaussians to approximate a few orthonormal columns. But with as many columns as $m\in\Omega(n)$, the approach is no longer valid. See Jiang, Tiefeng. "How many entries of a typical orthogonal matrix can be approximated by independent normals?" The Annals of Probability 34.4 (2006): 1497-1529.

Added numerical example
Source Link

Given fixed real symmetric $D\in\mathbb{R}^{n\times n}$ with $n$ distinct eigenvalues, let $U$ be a random orthogonal matrix selected uniformly from the space of $n\times n$ orthogonal matrices, and denote the first $m$ columns of $U$ as the rectangular matrix $Q\in\mathbb{R}^{n\times m}$.

We are interested in the quantity $$\mu=\lambda_\min(D+QQ^T) - \lambda_\min(D).$$$$\mu=\lambda_\min(D+QQ^T) - \lambda_\min(D)$$ Offor many columns, i.e. with $m\in\Omega(n)$. Of course, $0 \le \mu \le 1$. But given that $Q$ is randomly selected, the decoherence between the bases of $D$ and $Q$ causes the distribution of $\mu$ to lie strictly in between the two extremes. For sufficiently large $n$, choosing $m\in\Omega(n)$ columns the following statements should imply thathold:

  1. The event $\mu=0$ occurs with probability zero.
  2. The expectation $\mathbb{E}\{\mu\}$ is bounded from below by an absolute constant.
  3. The distribution of $\mu$ concentrates about $\mathbb{E}\{\mu\}$.

Indeed, all three statements are readily confirmed using numerical simulations. But how might we go about proving these statements?


Numerical example. We sample $\mu$ for $n\in\{10,30,100,300\}$ and $m=n/4$, performing 1000 trials for each $n$. In each trial, $D$ is set to be a diagonal matrix with random Gaussian entries. Numerical example


Remark 1. Since $D$ is fixed, we can, WLOG, assume that $D$ is positive definite and / or diagonal.

Remark 2. Many of the existing work on random orthogonal projections use standard Gaussians to approximate a few orthonormal columns. But with as many columns as $m\in\Omega(n)$, the approach is no longer valid. See Jiang, Tiefeng. "How many entries of a typical orthogonal matrix can be approximated by independent normals?" The Annals of Probability 34.4 (2006): 1497-1529.

Given fixed real symmetric $D\in\mathbb{R}^{n\times n}$ with $n$ distinct eigenvalues, let $U$ be a random orthogonal matrix selected uniformly from the space of $n\times n$ orthogonal matrices, and denote the first $m$ columns of $U$ as the rectangular matrix $Q\in\mathbb{R}^{n\times m}$.

We are interested in the quantity $$\mu=\lambda_\min(D+QQ^T) - \lambda_\min(D).$$ Of course, $0 \le \mu \le 1$. But given that $Q$ is randomly selected, the decoherence between the bases of $D$ and $Q$ causes the distribution of $\mu$ to lie strictly in between the two extremes. For sufficiently large $n$, choosing $m\in\Omega(n)$ columns should imply that:

  1. The event $\mu=0$ occurs with probability zero.
  2. The expectation $\mathbb{E}\{\mu\}$ is bounded from below by an absolute constant.
  3. The distribution of $\mu$ concentrates about $\mathbb{E}\{\mu\}$.

Indeed, all three statements are readily confirmed using numerical simulations. But how might we go about proving these statements?


Remark 1. Since $D$ is fixed, we can, WLOG, assume that $D$ is positive definite and diagonal.

Remark 2. Many of the existing work on random orthogonal projections use standard Gaussians to approximate a few orthonormal columns. But with as many columns as $m\in\Omega(n)$, the approach is no longer valid. See Jiang, Tiefeng. "How many entries of a typical orthogonal matrix can be approximated by independent normals?" The Annals of Probability 34.4 (2006): 1497-1529.

Given fixed real symmetric $D\in\mathbb{R}^{n\times n}$ with $n$ distinct eigenvalues, let $U$ be a random orthogonal matrix selected uniformly from the space of $n\times n$ orthogonal matrices, and denote the first $m$ columns of $U$ as the rectangular matrix $Q\in\mathbb{R}^{n\times m}$.

We are interested in the quantity $$\mu=\lambda_\min(D+QQ^T) - \lambda_\min(D)$$ for many columns, i.e. with $m\in\Omega(n)$. Of course, $0 \le \mu \le 1$. But given that $Q$ is randomly selected, the decoherence between the bases of $D$ and $Q$ causes the distribution of $\mu$ to lie strictly in between the two extremes. For sufficiently large $n$ the following statements should hold:

  1. The event $\mu=0$ occurs with probability zero.
  2. The expectation $\mathbb{E}\{\mu\}$ is bounded from below by an absolute constant.
  3. The distribution of $\mu$ concentrates about $\mathbb{E}\{\mu\}$.

Indeed, all three statements are readily confirmed using numerical simulations. But how might we go about proving these statements?


Numerical example. We sample $\mu$ for $n\in\{10,30,100,300\}$ and $m=n/4$, performing 1000 trials for each $n$. In each trial, $D$ is set to be a diagonal matrix with random Gaussian entries. Numerical example


Remark 1. Since $D$ is fixed, we can, WLOG, assume that $D$ is positive definite and / or diagonal.

Remark 2. Many of the existing work on random orthogonal projections use standard Gaussians to approximate a few orthonormal columns. But with as many columns as $m\in\Omega(n)$, the approach is no longer valid. See Jiang, Tiefeng. "How many entries of a typical orthogonal matrix can be approximated by independent normals?" The Annals of Probability 34.4 (2006): 1497-1529.

Source Link
Loading