Distribution of the constraint matrix conditioned on the solution of the linear system

Question

Suppose that A is a random matrix in $R^{n\times n}$, with each component independently and identically distributed (iid) according to $\mathcal{N}(0,1)$. Additionally, b is a random vector in $R^n$, with each component also iid according to $\mathcal{N}(0,1)$. We let $x$ denote the solution of the linear system $Ax = b$.

Let the direction of $x$ be denoted as $d = \frac{x}{\|x\|}$, where $\|x\|$ represents the $l_2$ norm of $x$.

Specifically, I'm curious whether the conditional distribution of $A|d$ is equivalent to the original distribution of $A$. It looks like they are the same because of distribution of $A$ and $d$ are both fully symmetric and centered at $0$

Answer 1

(It will be assumed here that $A$ and $b$ are independent. Of course, without an assumption on the joint distribution of $A$ and $b$, hardly anything can be said about the joint distribution of $A$ and $d$.)

Your question, in other words, is whether $A=[a_{i,j}]_{i,j=1}^n$ and $d=[d_1,\dots,d_n]^T$ are independent. This is of course so for $n=1$.

However, $A$ and $d$ are not independent already for $n=2$. Indeed, let $$R:=\frac{d_1^2}{d_1^2+d_2^2} =\frac{\left(r a_{2,2}-a_{1,2}\right)^2}{\left(r a_{2,1}-a_{1,1}\right)^2+\left(r a_{2,2}-a_{1,2}\right)^2},$$ where $r:=b_1/b_2$ and $b=[b_1,b_2]^T$, so that $r$ has the standard Cauchy distribution. So, the conditional expectation $$f(A):=E(R|A)$$ is the integral of a rational expression, so that $f$ is a certain elementary function, which is continuous where it is defined, and it is defined on an open set of full measure wrt the distribution of the random matrix $A$. Moreover, $f(A)=1/3$ if $A=\begin{bmatrix}2&1\\ 1&1\end{bmatrix}$ and $f(A)=1/4$ if $A=\begin{bmatrix}3&1\\ 1&1\end{bmatrix}$. Details of the calculation of $f(A)$ and its two particular values $1/3$ and $1/4$ just mentioned are given in this Mathematica notebook and its pdf image.

So, the distribution of the random variable $f(A)$ is nondegenerate. So, $$Ef(A)\frac{d_1^2}{d_1^2+d_2^2}=Ef(A)R=Ef(A)E(R|A)=Ef(A)^2 \\ > (Ef(A))^2 =Ef(A)\,ER=Ef(A)\,E\frac{d_1^2}{d_1^2+d_2^2},$$ which proves that $A$ and $d$ are not independent. $\quad\Box$

Answer 2

(It will be assumed here that $A$ and $b$ are independent. Of course, without an assumption on the joint distribution of $A$ and $b$, hardly anything can be said about the joint distribution of $A$ and $d$.)

Your question, in other words, is whether $A=[a_{i,j}]_{i,j=1}^n$ and $d=[d_1,\dots,d_n]^T$ are independent. This is of course so for $n=1$.

However, already for $n=2$ a numerical calculation suggests that $$E(A_{1,1}^2-1)d_1^2<-0.09,$$ so that $E(A_{1,1}^2-1)d_1^2\ne0=E(A_{1,1}^2-1)\,Ed_1^2$ and hence $A$ and $d$ are not independent.

---

Here are 10 (rounded) simulated values of $E(A_{1,1}^2-1)d_1^2$ using $10$ Monte Carlo samples each of size $10^6$ from the joint distribution of $A$ and $b$:
$$-0.123813, -0.125708, -0.125293, -0.125339, -0.124647, -0.125159, \ -0.125416, -0.125916, -0.124846, -0.124275.$$

These values can be compared with the value of the $6$-fold integral $E(A_{1,1}^2-1)d_1^2$ estimated by using the Mathematica command NIntegrate, which gave about $-0.125265\pm0.029215$.