I'm interested in bounding the tail probabilities of a quadratic form $x^t A x$ where $x\in \mathbb{R}^n$ is a sub-Gaussian vector with independent entries. $A\in \mathbb{R}^{n\times n}$ is a matrix. So I'm exactly in the setup of the Hanson-Wright inequality. In fact, I wish I could use it because if it would apply, it would give me exactly the bounds I'm looking for.

My problem is that in my case, the matrix $A$ is random, too. Worse even, I don't have independence of $A$ and $x$. However, there are two special properties in my case which can be used:

  1. $A$ and $x$ are uncorrelated, i.e. $\mathbb{E}[Ax]=\mathbb{E}[A]\mathbb{E}[\mathbb{x}]=\mathbb{E}[A]0=0$.
  2. $A$ is an orthogonal projection of rank $r < n$.

So my question is: Does somebody know a generalization of the Hanson-Wright inequality which would apply in this case?

[I am asking this question is because I study the finite-sample performance of OLS and other linear estimators. In the case of OLS, one can think of $A$ as orthogonal projection on the column space of the regressors and of $x$ as the error term. If the regressors were fixed, then Hanson-Wright would do the job immediately, but I need to allow for the regressors to be random, too.]

  • $\begingroup$ Uncorrelatedness alone is most likely insufficient for saying much. $\endgroup$ – passerby51 Jan 28 at 6:33

Let $x \sim N(0,I_n)$. For any independent rank-1 projection $A$, conditioned on $A$, we have $$x^T A x \sim \chi^2_1.$$ So unconditionally, $x^T A x = O(1)$ with high probability.

Now, let $A = \frac{x x^T}{\|x\|_2^2}$. Then, $A$ is a rank-1 projection and we have $\mathbb E [A x] = \mathbb E[x] = 0 = \mathbb E[A] \mathbb E[x]$. So, $A$ and $x$ are uncorrelated (in the sense stated in the question). But $$ x^T A x = \frac{x^T x x^T x}{\|x\|_2^2} =\|x\|_2^2 \sim \chi_n^2 $$ so $x^T Ax \approx n$ with high probability. (This rank-1 projection behaves like the full rank projection when applied to $x$.)

Assuming independence of $A$ and $x$, one can condition on $A$ and apply the Hanson--Wright inequality. Since the bound does not depend on $A$ (it only depends on $\|A\|_F = r$ and $\|A\| = 1$), the same bound would hold unconditionally. It would be as if $A$ was deterministic.

| cite | improve this answer | |
  • $\begingroup$ Wow, what a fantastic example! Would you expect a positive answer with independence of A and x? Or should I ask a new question for that and accept your answer straight away? I'm new here. $\endgroup$ – Puzzler Jan 28 at 7:36
  • $\begingroup$ @Puzzler, yes, the answer would be positive with independence. I will add a comment to the answer, no need for a new question. $\endgroup$ – passerby51 Jan 28 at 13:59
  • $\begingroup$ Thank you, @passerby51 ! I'll try to see whether I can justify independence. $\endgroup$ – Puzzler Jan 28 at 15:30
  • $\begingroup$ @Puzzler, You are welcome. $\endgroup$ – passerby51 Jan 28 at 15:59

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.