This question is migrated from math.stackexchange.

Let me first describe the problem I am trying to solve and then the question I have. I greatly appreciate anyone who can shine some light on it.

There are two parameters with the relationship as follows:

$a(x) = \int_{k_1}^{k_2} b(k)\exp(-kx)dk$

where $a(x)$ is the experimental data I have (quantized in x axis), and $b(k)$ is the information I want to get.

I am trying to solve the problem with such procedure:

Discretize $b(k)$ in k axis, so the above integral can be formatted as matrix multiplication like $a=Mb$, where the matrix $M$ contains the information of $\exp(-kx)dk$;

Apply zero-order (L0)

**Tikhonov regularization**with regularization parameter $\lambda$, and the problem can be formatted as $a=(M+\lambda I) b$ where $I$ is an identity matrix. The term $\lambda I$ has the meaning of solution smoothness. It is for damping the effect of experimental data noise on the final result $b(k)$ to avoid overfitting.Use some criterion (such as L-curve criterion http://www.sintef.no/upload/IKT/9011/SimOslo/vskoler/2005/notes/Lcurve.pdf) to determine the regularization parameter $\lambda$, and solve for $b(k)$ from the problem described by Step 2.

However, I find if I change the order of Tikhonov regularization to first-order (L1) or second order (L2), meaning the smoothness term in Step 2 becomes $\lambda I_1$ or $\lambda I_2$ where $I_1$ is the first-order derivative matrix and $I_2$ is the second-order derivative matrix, I can still get solution but it is very different to that from L0.

The difference is justified because L1 and L2 put more weight on the solution smoothness rather than fitting residual. But I do not have prior information about solution $b(k)$, so I do not know which solution I should use.

Can anyone provide any guide on choosing the order of such regularization?

someinformation (or at least intuition) about $b$: If not, why isn't $b = M^{-1} a$ (or $b=M^\dagger b$ if $M$ is not invertible) an acceptable solution? Of course you know, that $M$ is very ill-conditioned and hence, noise if $a$ would be amplified very much, but still: Without any assumptions on $b$, that would be a valid solution. $\endgroup$ – Dirk Nov 7 '13 at 16:33