5
$\begingroup$

In the chemical analysis by instruments, the signals of several molecules are overlapped which makes it difficult to determine the true area of each peak, such as those shown in red. I simulated this as a sum of six Gaussians (with some tailing elements)

  1. One of the simplest technique is to raise the discrete signal values to any positive power (n>0). The standard deviation of the Gaussians becomes smaller and smaller (C, in blue). The big drawback is that we lose all the original peak area information. The transformed data is highly resolved now at the cost of losing true area information.

  2. Alternatively, we can add a first derivative of the signal and subtract the second derivative from the original signal i.e., Sharpened signal= Original signal +K (first derivative) - J(second derivative)

K and J are small positive real numbers. This neat "trick" maintains the true area because area under the derivatives is negligible (zero in ideal cases).

Do mathematicians use any other transformations which can make each overlapping peak very narrow, yet maintain the original peak areas. I am not interested in curve fitting techniques at this moment. Any pointers to some similar "peak sharpening" transformations would be appreciated which can resolve overlapping signals.

Thanks.

enter image description here

$\endgroup$
9
  • $\begingroup$ related question: mathoverflow.net/q/334946/11260 $\endgroup$ Jun 29, 2019 at 10:43
  • $\begingroup$ Stupid question: are you interested in sharpening the peaks for the sake of it or what you really care about is just finding a good approximation to the individual peak areas (and, possibly, the centers)? Also, if I remember my physics right, the resonance peaks are Poisson rather than Gaussian ($\frac{A}{(x-x_0)^2+\omega^2}$). Am I confused here? $\endgroup$
    – fedja
    Jun 30, 2019 at 11:10
  • $\begingroup$ @fedja, it is valid question. I am interested in both. However, an "ideal" sharpening process should resolve the overlap and maintain the area of original peaks. Sorry, for the confusion, these are not spectroscopic resonance peaks but a simluation of light absorption of six components in a flow cell (=chromatography) by exponentially modified Gaussians. $\endgroup$
    – AChem
    Jun 30, 2019 at 14:25
  • $\begingroup$ So what it the true shape then? The best thing (IMHO) is an intelligent de-convolution (everything else that preserves areas, is translation invariant, etc. is just a more or less crude approximation of it) but the best multiplier to use certainly depends on the shapes of individual signals. $\endgroup$
    – fedja
    Jun 30, 2019 at 14:31
  • $\begingroup$ The true shape of a chromatographic peak is a pure Gaussian. However due to some flow effects, most of the real peaks are modeled by exponentially modified Gaussians (so called EMG peaks). What type of deconvolution are you suggesting? $\endgroup$
    – AChem
    Jun 30, 2019 at 16:12

3 Answers 3

5
$\begingroup$

OK, here are my 2 cents. I'll try to outline the logic and then make a conclusion. We'll start with the Gaussian case.

Suppose that we have a Gaussian peak $f(t)=e^{-t^2/2}$. We want to de-convolve it to the Dirac delta-measure. Since the Fourier transform $\widehat f(s)=\int f(t)e^{-ist}\,dx$ satisfies $\widehat{(f*g)}=\widehat f \widehat {g{,}}$ and the Fourier transform of the standard Gaussian is (up to a constant factor) the standard Gaussian again, the naive but natural idea is just to multiply $\widehat f$ by $e^{s^2/2}$ and to take the inverse Fourier transform of the resulting identically $1$ function.

Unfortunately this is unfeasible as written because of the noise, finite computational precision, etc., so we have to truncate the natural Fourier mulitplier $e^{s^2/2}$ somehow. The idea is to multiply it by some even positive cutoff function $\Psi$ with bounded support. Then the noise will be amplified (on the worst frequency) by $$ M=\max_s e^{s^2/2}\Psi(s) $$ but instead of the Dirac point mass, we will get the inverse Fourier transform of $\Psi$. We want that inverse Fourier transform to be as concentrated near the origin as possible. We also want to make it a non-negative function rather than some sign-changing wave. Since we must have $\Psi(0)=1$ to preserve the integral of $f$, we come to the problem of choosing an even function $\Psi$ with bounded support (say $[-\pi,\pi]$; we can always scale later) such that $\Psi(0)=1$, the inverse Fourier transform $\Psi^\vee\ge 0$, and $-\Psi''(0)=\int\Psi^\vee(t)t^2\,dt$ is minimized (other measurements of concentration give slightly different optimizers but pretty much the same results).

This problem is solvable and the solution is the (properly normalized) convolution of $\chi_{[-\pi/2,\pi/2]}(s)\cos s$ with itself, i.e., $$ \Psi(s)=\frac 1\pi[(\pi-|s|)\cos s+\sin|s|] $$ (I'll skip the derivation). Thus, after various scalings, the final family of multipliers is $$ H_{\alpha,K}(s)=e^{\alpha^2 s^2/2}\Psi(K^{-1}\alpha s)\,. $$ and the "peak sharpening formula" is $f\mapsto (H_{\alpha,K}\widehat f)^\vee$.

It turns out that $K=1$ does not noticeably change the picture (but filters out the high frequency noise from the measurements). The noise amplification grows pretty fast with $K$. The corresponding maxima are about $$ \begin{matrix} K &\max \\ 1.0 & 1.09 \\ 1.1 & 1.55 \\ 1.2 & 2.65 \\ 1.3 & 5.33 \\ 1.4 & 12.4 \\ 1.5 & 33.2 \end{matrix} $$ (with noise at $1$ to $10\%$ of the signal, the higher values of $K$ are pretty useless), so I would recommend setting $K$ between $1.3$ and $1.4$. The whole game is choosing $\alpha$. The legitimate choice is up to the width of the narrowest peak but somewhat higher values are also sort of admissible except you'll get peaks exchanging energy a bit. The rule of thumb is to watch for noticeable negative values in the transformed signal: once they appear, you certainly went too far. Before that moment you are reasonably safe (but not entirely foolproof).

The only difference for EMG is that you want to add one more factor $1+i\beta s$ to your multiplier, i.e., to play with the family $$ H_{\alpha,\beta, K}(s)=e^{\alpha^2 s^2/2}(1+i\beta s)\Psi(K^{-1}\alpha s)\,. $$ If $\beta>0$ gets large, it may force you to make $K$ smaller. The effect of going too far in $\beta$ is the same: you'll see noticeable negative values in the transform plus the peaks will noticeably shift to the left. Playing with $2$ parameters is harder than with one and I have no clear advice on in which order to adjust them to get the best results. Just try and let me know what you think.

Successful story.

The original signal is in green, the dots are the actual locations at which EMG's are "centered". Something like that is possible IF you can guess one particular parameter right. I still cannot figure out how to teach the machine that guessing.

You'll, probably, like this one too:

enter image description here

Or this one:

enter image description here

The current version of the code. This is a dirty homemade contraption, of course, but it is totally automatic.

import graph;
access settings;
//settings.outformat="png";


size(500,300,IgnoreAspect);

int sec=seconds();
srand(sec); //Just generates a new picture every time
//srand(1562245957); //was really bad, fixed
//srand(1562264140); //was bad, fixed
//srand(1562265837); //still bad

bool ON=false; //Set to true if you want to see suppressed low values

real beta=0.04; //Noise level (maximum; you need to adjust the code a bit to use the average)
real Beta=0.12; //Cutoff level (approximately 3 times maximal noise)

real K=1.7, L=3, TOL=2.5;  //Don't change these UNLESS
int res=8, dirr=5;//you understand what you are doing


int M=4; //The number of peaks. Set to 1 to see the individual response.

int n=4096; //The sampling range (4096 time ticks with no signal before and after)
real h=1/n;

real QQ=0.07*unitrand(); //Average lambda; set to 0 to see pure Gaussians
real AA=0.15*sqrt(unitrand()); //Average sigma; set to zero to see pure exponentials
real det=unitrand(); //Peak spacing from equidistant det=1 to random det=0

real[] A,Q,X,Y;
for(int m=0;m<M;++m)
{
A[m]=AA*(0.2+unitrand());
Q[m]=QQ*(0.5+0.5*unitrand());
X[m]=det*m/M*n/2+(1-det)*(rand()%quotient(n,2))+quotient(n,8);
Y[m]=0.45+0.55*unitrand();
}

X=sort(X); A=sort(A); //sigma grows with time

real qual(real x)
//deconvolution quality control: the bigger the sum of qual(p[k]), the better.
{
return 
max(x-2*Beta,0)^3-min(x,0)*Beta^2;
}


//write(A); write(Q); pause();

pair[] p, ptemp; 
//p is the (simulated) raw data. It would be interesting to try it on something real.

////////////////Generation of p /////////////////

for(int k=0;k<n;++k) {p[k]=(0,0); ptemp[k]=(0,0);}
p.cyclic=true; ptemp.cyclic=true;

for(int m=0;m<M;++m)
{
pair f(int k)
{ 
return exp(-pi*A[m]^2*k^2)*expi(-2*pi*k*X[m]*h)/(1,2*pi*Q[m]*k);
}
for(int k=-floor(n/2);k<n/2;++k) ptemp[k]=f(k);
ptemp=fft(ptemp);
ptemp/=max(map(abs,ptemp));
ptemp.cyclic=true;
p+=Y[m]*ptemp;
}

real s=0;
for(int k=0;k<n;++k) {s+=p[k].x;}

for(int k=0;k<n;++k) p[k]+=beta*(2*unitrand()-1);


//////////// End of generation of p ///////////////////

//////////// Algorithm begins ///////////////////

pair[] g=fft(p)/n; 

real psi(real t)
{
real s=abs(t);
if(s>pi) return 0;
else 
//return sin(s)/s*(1-s/pi);
return ((pi-s)*cos(s)+sin(s))/pi;
}


pair R(pair x) 
//Suppression of low values. 
{
if (ON) return x;
real y=max(x.x-Beta,0);
if(x.x<0) return (0,0);
else return ((x.x<4*Beta? y*4/3:x.x),x.y)/(1-4*Beta^2);
}

real expp(real x)
{
return 1+x+x^2/2+x^3/12;//A bit more stable than the true exp
return exp(min(x,15));
}


struct T {pair[] p; real u; real q;}

int count=0;

real[] pM; pM[n]=0;
for(int k=n-1; k>-1; --k) pM[k]=max(p[k].x,pM[k+1]);

pair[] pp=copy(p), p1;

T U(pair D, real qq)
{
++count;
write("U "+string(count));



real u=0,v=1;

void proc()
{
for(int kk=0; kk<10;++kk)
{

real w=(u+v)/2;
real a=w*D.x, q=w*D.y;

real b=min(max(a*sqrt(2*pi)/K,2*pi*q/L),1/5);


pair[] G=fft(pp)/n, mult;
p1=copy(pp);

for(int k=0;k<n;++k) mult[k]=(0,0);
mult.cyclic=true;

G.cyclic=true;

for(int k=floor(-n/2);k<n/2;++k) 
{
mult[k]=expp(pi*a^2*k^2)*(1,-2*pi*q*k)*psi(b*k);
if(abs(mult[k])>TOL) mult[k]=TOL*unit(mult[k]);
G[k]*=mult[k];
}

pair[] ppp=fft(G); ppp=reverse(ppp);
ppp.cyclic=true;

bool flag=true;

//for(int k=0;k<n;++k) if(abs(mult[k])>TOL) flag=false; 

for(int k=floor(n*qq-n/res);k<n+n/2/res && flag;++k)
{
if(
ppp[k].x<-0.8*Beta || (k>-1 && k<n && pM[k]<Beta/2 && ppp[k].x>Beta)
) 
flag=false; 
}

if(flag && abs(G[floor(n/2)-1])<1/10^10) {u=w; p1=copy(ppp);} else v=w;
}
}

proc();
p1.cyclic=true;


if(p1[n-1].x>Beta) {write(p1[n-1].x,pM[n-1]); pause();}

real q=0;
for(int k=floor(n*qq-n/res);k<n;++k) q+=qual(p1[k].x); 

T tt=new T;
tt.p=copy(p1); tt.u=u; tt.q=q;

write(tt.u);

return tt;

}




T[] tt;

for(int k=0; k<res; ++k)
{
draw(box((k/res,0),((k+1)/res,1)),yellow); 
pair D=expi(pi/2);
tt[k]=U(D,k/res); real W=tt[k].q;
for(int m=dirr-1; m>-1; --m)
{
D=dir(90*m/dirr);
T ttt=U(D,k/res);
if(ttt.q>W) {W=ttt.q; tt[k]=ttt;}
}
//write (tt[k].u, tt[k].q);
pp=copy(tt[k].p);
}


for(int k=0;k<n;++k)
{
int q=floor(k/n*res);
if(q<1) {pp[k]=tt[0].p[k];}
else
{
real t=q+1-k/n*res;// t=sin(pi*t/2);
pp[k]=((1-t)*tt[q].p[k]+t*tt[q-1].p[k]);
}
}

pp=map(R,pp);
real ss=0;
for(int k=0;k<n;++k) {ss+=pp[k].x;}

write("*************");



path Q=nullpath,  P=nullpath;
for(int k=0;k<n;k+=4) 
{
Q=Q--(k*h,pp[k].x); 
P=P--(k*h,p[k].x); 
}

draw(P,green);
draw(Q);
draw((0,1)--(0,0)--(1,0));
draw(box((0,-Beta),(1,Beta)),red);


for(int m=0; m<M ;++m)
{
dot((X[m]*h,0)); 
pair Z=intersectionpoint((X[m]*h,-20*M)--(X[m]*h,20*M),Q),ZZ=(X[m]*h,Y[m]);
draw((X[m]*h,0)--Z,blue); dot(ZZ,magenta);
}




label(string(sec),(0,0),S);



label("Area ratio="+string(floor(1000*ss/s)/1000),(0.8,1.13),(0,0),fontsize(8));


//pause();
$\endgroup$
24
  • $\begingroup$ fedja, If I understand your suggestion correctly, basically you wish to broaden the FT of the original signal. Once it is broadened, one can do an inverse transform and recover the "sharpened" peak. Is this the main thrust of this idea? And, could you define what "s" is in terms of time? $\endgroup$
    – AChem
    Jul 1, 2019 at 15:11
  • 1
    $\begingroup$ @M.Farooq Yeah, something like that though I really want to divide by an appropriate function rather than just "broaden". I tried to simulate it with and without noise. It works, especially if you combine it with some transformation that merely cuts small values to ignore the stuff that appears on the bottom (like 5-10% of the maximum) and preserves areas up to 5-15% error but it is pretty hard to choose parameters automatically: it tends to overshoot a bit, which is OK, but it also tends to de-convolve with the exponential part even if it is not there. I'll post some pictures, if you want. $\endgroup$
    – fedja
    Jul 1, 2019 at 18:22
  • $\begingroup$ @M.Farooq $s$ is the Fourier variable ("frequency"). Time is $t$. $\endgroup$
    – fedja
    Jul 1, 2019 at 18:23
  • $\begingroup$ fedja, I would be happy to see the figures. Just curious. $\endgroup$
    – AChem
    Jul 1, 2019 at 18:52
  • $\begingroup$ @M.Farooq I posted a "good" one. Failures are also pretty common if the parameters are chosen incorrectly. $\endgroup$
    – fedja
    Jul 2, 2019 at 19:58
2
$\begingroup$

Here is an overview of peak sharpening (resolution enhancement) techniques:

1. even derivative sharpening: $R=Y-k_2 Y''+k_4 Y''''$: works well for symmetric line shapes, reduces the width of a Gaussian peak by 20% and of a Lorentzian peak by 60%

2. first derivative symmetrization: $R=Y+k_1 Y'$: converts an exponentially broadened peak into a symmetric Gaussian, and then the width can be reduced via method 1.

3. power law method: raise each data point to a power $n>1$: moves the peaks apart, increasing the resolution, to be followed by sharpening method 1.

4. deconvolution: effective if the original shape of the peaks has been broadened and/or made asymmetrical by a broadening process that is known or can be measured separately.

$\endgroup$
1
  • $\begingroup$ Dear Carlo, Thanks, I am familiar with these ones, the author of this website is my co-author on these methods. I am trying to collect the names of more approaches, if any besides 1-4. $\endgroup$
    – AChem
    Jun 29, 2019 at 12:19
2
$\begingroup$

There are numerous methods in the context of "inverse problems". Assuming that the Gaussian peaks are generated by a convolution of a "spike train" (i.e. a sum of positive Diracs peaks) with a Gaussian (plus some noise), you can do any of the numerous methods for regularized deconvolution.

Let me just mention variational regularization: Let $g$ be the Gaussian and $\mu$ be the spike train. Then your signal is $f=g*\mu + $ noise. A regularized deconvolution can be obtained by minimizing $$\|g*\mu - f\| +\alpha\|\mu\|$$ but the choice of norms and the regularization parameter $\alpha>0$ really matter. A simple choice is the $L^2$ norm for the residual and the variation norm (aka Radon norm) for the measure $\mu$. This provably leads to a spiky reconstruction and the process is stable.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.