6
$\begingroup$

Let $f: \mathbb{R} \rightarrow \mathbb{R}$ be a continuous periodic function. If we replace $f$ by $$g(t) := f(\frac{1}{\lambda}t)$$ for some $\lambda > 0$, this new function $g$ has period $\lambda$ times the period of $f$, and$\mathbf{{}^1}$ $$\hat{g}(s) = \lambda \hat{f}(\lambda s)$$.

That is $f$ has been pitch-shifted down by $\log_2 \lambda$ octaves, but also time-dilated by a factor of $\lambda$.

My question is: is there mathematically well-defined operation of pitch shift without time dilation?

I strongly suspect the answer is "no", because any continuous linear operation $T_{\lambda}$ which correctly pitch-shifts down by $\log_2 \lambda$ octaves all functions of the form $f(t) = sin(\omega t)$, i.e., for which $T_{\lambda}(sin( \omega t)) = sin(\frac{\omega}{\lambda} t)$, will necessarily have $T_{\lambda}(f(t)) = f(\frac{1}{\lambda}t)$ for every continuous periodic function $f$.

This doesn't immediately rule out a non-linear (or possibly non-continuous) operation, so I'm wondering if there is anything in the literature about such an operation, or a "no-go" theorem showing that the notion of pitch-shift without time-dilation is incoherent, period.

Most papers about this subject are written by engineers, who really only care about performing this operation approximately to give more-or-less the correct psycho-acoustic effect. E.g. this paper using wavelets, which I take to be roughly state-of-the-art. Other past approaches have been even worse: e.g. breaking up a signal into short time intervals doing a naive pitch-shift with time dilation (possibly repeating or clipping), and then assembling them back together, introducing discontinuities when moving from one time interval to the next, which are then smoothed over by some filter.

My motivation here is to be able to have an ideal standard by which to judge the degree of accuracy of a pitch-shift algorithm. If there is none, then the question of the accuracy of a pitch-shift algorithm is an empirical question about subjective experience rather than a mathematical one.


$\mathbf{{}^1}$ The Fourier transform will have to be a distribution rather than a function.

$\endgroup$
4
  • 1
    $\begingroup$ As you point out, the pitch shift is a time dilation, so if your question is "is it possible to get the same result but with a function which is not a time dilation?", then the answer is 'no' by definition. Is your question whether there is some definition that apparently doesn't involve time dilation, that nonetheless produces the same result? $\endgroup$
    – LSpice
    Mar 26 '18 at 18:17
  • 3
    $\begingroup$ I'm asking if there is a mathematical analogue of the intuitive operation of "pitch shifting", which you can find approximately implemented in many audio-editing programs. Ideally, a pitch-shift down without time-dilation would have the following property: if you record an hour-long orchestra concert, the pitch-shifted version of that concert will still be an hour-long, still have just as many notes played at the same tempo, but the violins would sound like violas, the violas like cellos, and the cellos like basses. This operation seems well-defined, as musicians can do it. But is it really? $\endgroup$
    – James
    Mar 26 '18 at 18:34
  • 2
    $\begingroup$ I am pretty sure an argument can be made, via the Gabor limit (a version of the uncertainty principle), to rule out the existence of a "mathematically perfect" pitch shift. On the other hand, for questions relating to audio signals, human beings' hearing has a finite bandwidth, so if you are willing to formulate things in terms of signals limited between to, say, 20 and 20000 Hertz, then maybe something reasonable can be said. $\endgroup$ Mar 26 '18 at 19:36
  • $\begingroup$ Definite +1, I do a lot of music editing, and I've always wondered how they did it while not disobeying the bare laws of Fourier Analysis. But if you pay real close attention to vocal samples tuned up or down time invariant, you'll notice that certain words get spoken at slightly different times. $\endgroup$
    – user78249
    Mar 27 '18 at 2:15
3
$\begingroup$

Let us model the signal $f$ as follows: \begin{equation} f(t)=\Re\sum_{j=1}^n c_j(t)K((t-t_j)/\tau_j)e^{i\omega b_j t} \end{equation} for $t\in[0,T]$, where $n$ is not very large, $\omega$ is a large positive real number, $b_j\asymp 1$ ($b_j\in\mathbb R$), $K$ is a kernel function (such as $K(x)=e^{-x^2}$), the $t_j$'s are time moments in the interval $[0,T]$, $\tau_j\asymp1$ ($\tau_j\in\mathbb R$), and the $c_j$'s are complex-valued "amplitude" functions, which are slowly varying as compared with the harmonics $e^{i\omega b_j \cdot}$, except maybe only one of the harmonics.

I think what human beings may be doing is as follows: Locally, over time intervals that are small enough for the amplitudes to noticeably change and yet much greater that $2\pi/\omega$, using measurements of $f(t)$ at sufficiently many time moments $t$, they obtain estimates $\hat c_j$, $\hat K$, $\hat t_j$, $\hat\tau_j$, $\hat\omega$, $\hat b_j$ of $c_j$, $K$, $t_j$, $\tau_j$, $\omega$, $b_j$ and then pitch-shift, to get the transformed signal over the same time interval $t\in[0,T]$, of the same tempo, with as many "notes": \begin{equation} \hat f_\lambda(t)=\Re\sum_{j=1}^n \hat c_j(t)\hat K((t-\hat t_j)/\hat \tau_j)e^{i\hat \omega \hat b_j t/\lambda}. \end{equation} This does not look pretty, and yet it seems a reasonable way for the brain to act.

Here is an illustration of this pitch-shifting, with $\omega=30$ and $\lambda=2$:

enter image description here

$\endgroup$
4
  • $\begingroup$ FWIW, this is also similar to how many of the non-time-stretching pitch-shifting applications work; IIRC the magic phrase for the idea is granular synthesis. $\endgroup$ Mar 26 '18 at 21:57
  • $\begingroup$ This seems plausible (and seems pretty similar to the "wavelet" approach). I guess the idea would be to model the cochlea, and perhaps we would find that the intensity of stimulation at particular location at a particular time would correspond to an atomic qualitative aspect of the sound. Then "pitch shifting" would correspond to creating this same activation pattern translated and scaled along the cochlea (which we can imagine is a horn). Do you have a reference to a derivation from a physical model of human perception to this kind of procedure? $\endgroup$
    – James
    Mar 26 '18 at 22:49
  • $\begingroup$ I had never thought of or seen this problem before. At this point, I just thought that this could be a plausible way for the brain to deal with changing only the (compartively high) "local" frequency while keeping the "global" features of the original signal largely intact. $\endgroup$ Mar 27 '18 at 0:10
  • $\begingroup$ If only you modeled this in the 70s, you would've made a fortune. $\endgroup$
    – user78249
    Mar 27 '18 at 2:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.