Hanson-Wright inequality with random matrix

Question

I'm interested in bounding the tail probabilities of a quadratic form $x^t A x$ where $x\in \mathbb{R}^n$ is a sub-Gaussian vector with independent entries. $A\in \mathbb{R}^{n\times n}$ is a matrix. So I'm exactly in the setup of the Hanson-Wright inequality. In fact, I wish I could use it because if it would apply, it would give me exactly the bounds I'm looking for.

My problem is that in my case, the matrix $A$ is random, too. Worse even, I don't have independence of $A$ and $x$. However, there are two special properties in my case which can be used:

1. $A$ and $x$ are uncorrelated, i.e. $\mathbb{E}[Ax]=\mathbb{E}[A]\mathbb{E}[\mathbb{x}]=\mathbb{E}[A]0=0$.
2. $A$ is an orthogonal projection of rank $r < n$.

So my question is: Does somebody know a generalization of the Hanson-Wright inequality which would apply in this case?

[I am asking this question is because I study the finite-sample performance of OLS and other linear estimators. In the case of OLS, one can think of $A$ as orthogonal projection on the column space of the regressors and of $x$ as the error term. If the regressors were fixed, then Hanson-Wright would do the job immediately, but I need to allow for the regressors to be random, too.]

Answer 1

Let $x \sim N(0,I_n)$. For any independent rank-1 projection $A$, conditioned on $A$, we have $$x^T A x \sim \chi^2_1.$$ So unconditionally, $x^T A x = O(1)$ with high probability.

Now, let $A = \frac{x x^T}{\|x\|_2^2}$. Then, $A$ is a rank-1 projection and we have $\mathbb E [A x] = \mathbb E[x] = 0 = \mathbb E[A] \mathbb E[x]$. So, $A$ and $x$ are uncorrelated (in the sense stated in the question). But $$ x^T A x = \frac{x^T x x^T x}{\|x\|_2^2} =\|x\|_2^2 \sim \chi_n^2 $$ so $x^T Ax \approx n$ with high probability. (This rank-1 projection behaves like the full rank projection when applied to $x$.)

---

Assuming independence of $A$ and $x$, one can condition on $A$ and apply the Hanson--Wright inequality. Since the bound does not depend on $A$ (it only depends on $\|A\|_F = r$ and $\|A\| = 1$), the same bound would hold unconditionally. It would be as if $A$ was deterministic.