0
$\begingroup$

In their classic paper "Ideal spatial adaptation by wavelet shrinkage" (http://biomet.oxfordjournals.org/content/81/3/425.short?rss=1&ssource=mfr), Donoho and Johnstone make the following statements:

Let $f$ be a piecewise polynomial function. If the list of true discontinuities in $f$ was known via an oracle, we could use a least squares estimate for each of the segments to obtain an estimate $\hat f$ with ideal risk $\sigma^2 L(D+1)$, where $L$ is the number of segments and $D$ the degree of the polynomial. They call this "piecewise polynomial adaptation".

Later on, they consider a hypothetical method called "selective wavelet reconstruction". For each wavelet function $W_{j,k}$, and the wavelet coefficients

$w_{j,k} = \theta_{j,k} + z_{j,k}$

where $\theta$ and $z$ denote the signal and noise part of the coefficient, this method obtains a subset $\delta^\star$ of wavelet coefficient indices $(j,k)$ from an oracle such that a wavelet is in the list iff $\theta_{j,k} \neq 0$. In other words, it keeps those wavelets that span a discontinuity in $f$, hence removing all coefficients that are due to noise only.

They then go on to claim that

because of the orthogonality of the $(W_{j,k})$, $\sum_{(j,k)\in \delta^\star} w_{j,k} W_{j,k}$ is the least-squares estimate of $f$ and [the risk] $R(T(y, \delta^\star), f) = n^{-1}\{\#(\delta^\star)\}\sigma^2$

I do understand how this risk is derived (the retained coefficients are still contaminated by noise), but I don't see why selective wavelet reconstruction would be an LSE? The risk they derive for this method is similar, but not equal to the one for polynomial adaptation, which clearly IS an LSE.

I am clearly missing something here. Could somebody please explain how one would derive the second method to yield an LSE, and why that would be different from polynomial adaptation via least squares?

Improve this question
$\endgroup$
10
  • $\begingroup$ You didn't explain what is "the signal and noise part". I could take 1 hour for reading the paper, or you could explain. $\endgroup$
    – reuns
    Commented Nov 27, 2016 at 7:34
  • $\begingroup$ Certainly :-) The wavelet transform is linear, so it acts on the signal and noise part of the data seperately. More specifically, for data $x = f+\epsilon$, where $f$ is the signal and $\epsilon$ is i.i.d. centered Gaussian noise, and a wavelet function $\psi_{j,k}$, $w_{j,k} = \langle x, \psi_{j,k}\rangle = \langle f+\epsilon, \psi_{j,k}\rangle = \langle f, \psi_{j,k}\rangle + \langle \epsilon, \psi_{j,k}\rangle = \theta_{j,k} + z_{j,k}$. $\endgroup$
    – user32849
    Commented Nov 27, 2016 at 10:31
  • $\begingroup$ in eq (4) what is understand is that we have a signal model $F(n )= f(n) + e(n)$ where $e(n)$ is i.i.d. Gaussian and $f(n) = \sum_i p_i(n) 1_{n \in T_i}$ where $p_i$ are some polynomials ($f$ is piecewise polynomial) and and we want to LSE $f(n)$ from $F(n)$ given an oracle telling us $T_i$, so that we can use the LSE polynomial regression on each $P_i(n) =p_i(n)+e(n)$ to recover an estimate $\hat{p_i}(n)$ of $p_i(n)$ and a global estimate $\hat{f}(n) = \sum_i \hat{p_i}(n) 1_{n \in T_i}$ whose square error $\|\hat{f}-f\|^2$ will be much better than without the oracle. $\endgroup$
    – reuns
    Commented Nov 27, 2016 at 10:46
  • $\begingroup$ Then it describes a wavelet implementation of this, but I don't follow it. Can you give the clues ? It computes the WT of $F(n)$ and keeps only the main coefficients ? $\endgroup$
    – reuns
    Commented Nov 27, 2016 at 10:51
  • $\begingroup$ It keeps those coefficients that span across the discontinuities. If we know the breakpoints from an oracle, an LSE for each of the piecewise polynomials yields an ideal risk of $\sigma^2L(D+1)$. My problem is with the claim that an oracle with knowledge of $f$ that gives you an optimal subset of wavelet coefficients, this somehow also yields an LSE (why?), but for some reason that LSE has a different ideal risk. So two different oracles for two different methods both yield an LSE, but have different ideal risk? $\endgroup$
    – user32849
    Commented Nov 27, 2016 at 10:54

0

Reset to default

You must log in to answer this question.

Browse other questions tagged .