Reassigned spectrograms -- preliminary results!

[Paul L]

If you vary the Window size in spectrogram preferences, you soon discover the tradeoff between time resolution (which is better with smaller windows) and frequency resolution.

The "reassignment" method for sharpening spectrogram displays is one way to escape the tradeoff between time and frequency resolution, and it can be made even more precise with zero-padding of the windows. In broad terms: perform FFT of waveform data, as now. The resulting coefficients contribute to brightness of pixels in the display. But by using two other FFTs, two corrections can be calculated, one horizontal and one vertical, to move the contributions from the first FFT to better sample times and frequency bins, before remapping that to the grid of pixels. Thus some pixels may receive contributions from more than one window and bin of the first FFT, and others may receive none.

Here is the reference I am consulting. http://perso.ens-lyon.fr/patrick.flandrin/0065-Ch05.pdf


I have only a partial implementation working, that performs the vertical corrections only, but already the results are striking.

Here is the Spectrogram Preferences dialog in my project. Note the "zero padding" drop-down, which can choose among powers of 2, and the "Reassignment" checkbox. With these settings, the new feature does not affect the display:

Preferences.jpg (78.03 KiB) Viewed 765 times

Here is a five-second chirp, from 20 to 2000 Hz, linear frequency ramp, amplitude 0.8, waveform is square no alias, viewed with settings as above. There is also a narrow click in the middle, made by moving just one sample with the pencil tool.

Spectrum.png (410.26 KiB) Viewed 765 times

Here is the same with "Zero padding factor" increased to 8. There is eightfold better resolution along the vertical axis but not really more detail. Notice near the lower right that the lobes of the Blackman-Harris window are better resolved as ripples between the overtones. The waveform is still windowed only 256 samples at a time for each column of pixels, but that is inserted in a window of 2048 samples before FFT, giving more frequency coefficients. Thus too the time resolution of the click is no less precise.

Zero padding 8.png (896.21 KiB) Viewed 765 times

The first spectrum, reassigned (vertically). Notice that the overtones are more precisely indicated but the click is not any more smeared horizontally!

Reassigned.png (302.61 KiB) Viewed 765 times

More pictures to come...

[Paul L]

Reassignment and padding factor 8 yield this picture:

Zero padding 8 reassigned.png (746.74 KiB) Viewed 762 times

A zoomed view of the last:

Zoomed.png (273.13 KiB) Viewed 762 times

We see that when frequency resolution is very fine, the zoomed-out view may no longer indicate peak frequencies in white. That is because, when the vertical extent of one pixel covers more than one frequency bin, the display algorithm takes an average. Should it take a maximum instead?