-2
$\begingroup$

I'm trying to solve the following least squares problem:

$\underset{x}{\text{min}} ||Ax - \tilde{b}||_2$

where $Ax = b$ and $\tilde{b} = b + w$

Question:

How do I determine which probability distribution fits $w$ best?

Also, $A \in \mathbb{R}^{n\times n}$ is a large, and sparse Toeplitz matrix, $\tilde{b} \in \mathbb{R}^{n \times 1}$

$w$ is not your typical measurement noise and is probably not Gaussian. Also, I have access to $b$ (training data) and therefore, $w$.

Also, in a related question, I'm using LSQR to determine $x$ --- since I have my training data, is there any way to do cross-validation to determine how to best solve for $x$?

For example, I tried Tikhonov regularization to find the best $\lambda$ but I'm looking for a better method.

$\endgroup$
3
  • 1
    $\begingroup$ I think you should reformulate your question. it seems like a poorly formulated homework question. $\endgroup$ Jul 28, 2010 at 17:16
  • $\begingroup$ @Robin: It's not homework. Which part is poorly formulated? $\endgroup$
    – Jacob
    Jul 28, 2010 at 17:27
  • $\begingroup$ To start with, you could improve the question : "Given a vector w, how do I determine which distribution fits it best?" distribution of what ? what is "it". $\endgroup$ Jul 28, 2010 at 18:16

1 Answer 1

2
$\begingroup$

There's no way to answer in general what distribution data has. It has whatever distribution it has, but what you really want to know is whether a distribution which someone has identified adequately fits your data. So you have to propose a specific distribution first, then test for goodness of fit. Searching on "goodness of fit" will give you lots of resources, but a good general-purposed goodness of fit test is Kolmogorov-Smirnov, often abbreviated K-S.

You could look at a histogram of your data to start your search. Is it symmetric or skewed? Long tails or short tails? Unimodal or bimodal? If you post an image, someone may be able to suggest some distributions to test.

Your question may be more appropriate for the Statistical Analysis Stack Exchange site.

$\endgroup$
3
  • $\begingroup$ @John: Thank you! "goodness of fit" seems to be a good place to start. $\endgroup$
    – Jacob
    Jul 28, 2010 at 18:09
  • $\begingroup$ @jacob: another good place to start is trying to improve the way you formulate your question. $\endgroup$ Jul 28, 2010 at 18:18
  • $\begingroup$ @Robin: Thank you for the suggestion, how does it look now? $\endgroup$
    – Jacob
    Aug 6, 2010 at 15:55

Not the answer you're looking for? Browse other questions tagged or ask your own question.