User tony bruguier - MathOverflow most recent 30 from http://mathoverflow.net 2013-05-19T19:38:45Z http://mathoverflow.net/feeds/user/10043 http://www.creativecommons.org/licenses/by-nc/2.5/rdf http://mathoverflow.net/questions/42088/inequality-constrained-linear-regression-what-is-the-covariance-of-the-estimator Inequality-constrained linear-regression, what is the covariance of the estimator? Tony Bruguier 2010-10-13T23:45:45Z 2010-10-14T11:52:52Z <p>If you do a linear regression: $||Ax - e ||^2$, where e is iid Gaussian, mean 0 and variance 1, then your answer is $x_{hat} = (A' A)^{-1} (A' * e)$ and the covariance of $x_{hat}$ is $(A' A)^{-1}$</p> <p>Now, what if I add the linear inequality constraints $Bx > c$? There are algorithms that find the answer for a given $e$, but what is the covariance matrix?</p> <p>It seems like a non-trivial problem: <a href="http://www.gurulib.com/_user_manual_file/pic_1247578519497.pdf" rel="nofollow">http://www.gurulib.com/_user_manual_file/pic_1247578519497.pdf</a></p> <p>However, the author seems to give up: "A much more interesting problem is to analyze a properly truncated variance-covariance matrix of $b*$. However, it is beyond the scope of this paper."</p> <p>Of course, I can do a Monte-Carlo simulation, but a closed-form solution would be better. Any hint or reference?</p> <p>Thanks in advance, Tony</p> http://mathoverflow.net/questions/41756/making-matlab-svd-robust-to-transpose-operation/42089#42089 Answer by Tony Bruguier for Making MATLAB svd robust to transpose operation Tony Bruguier 2010-10-13T23:53:21Z 2010-10-13T23:53:21Z <p>In Matlab, there is a sign ambiguity (i.e. $\pm1$). In LAPACK, it is worse, the ambiguity is $e^{i \phi}$, which Matlab removes later on.</p> <p>Maybe you can take each vector of $U$ individual, find the coefficient with the largest absolute value, and make sure that this coefficient is always positive.</p> <p>This problem has never bothered me before though.</p> http://mathoverflow.net/questions/42088/inequality-constrained-linear-regression-what-is-the-covariance-of-the-estimator/42135#42135 Comment by Tony Bruguier Tony Bruguier 2010-10-19T19:01:20Z 2010-10-19T19:01:20Z Thanks for the reference. I have been reading it, but I am not done yet. Just so that I make sure I understand correctly, the approach still uses a random number generator, right? It's just a smart way to go at it. http://mathoverflow.net/questions/42088/inequality-constrained-linear-regression-what-is-the-covariance-of-the-estimator/42135#42135 Comment by Tony Bruguier Tony Bruguier 2010-10-14T14:27:22Z 2010-10-14T14:27:22Z Brian and Simon, Thanks for your answers. If the constraints are very narrow, then the vector $\hat{x}$ will most likely behave like a binary variable. If the constraints are very loose, then it $\hat{x}$ will behave like a normal variable. Sorry, I don't know how you would use Bayesian approach to solve this problem. Could you enlighten me? Also, please note that I am not interested in the pdf of $\hat{x}$, just its covariance matrix. I understand that a canned answer may or may not exist. The paper I reference seems to indicate that in 1976, there was no such answer. http://mathoverflow.net/questions/42088/inequality-constrained-linear-regression-what-is-the-covariance-of-the-estimator Comment by Tony Bruguier Tony Bruguier 2010-10-14T04:46:13Z 2010-10-14T04:46:13Z Well, the vector $x$ is random right? I agree it's not normal, but there's still a covariance matrix attached to it.