NoiseRemoval.cpp

[Paul L]

Nobody has mentioned performance yet. Better results might be bought with more compute time, and advanced settings might control that.

Now we use 2048 sample windows stepped by 1024. That ratio needs to be a power of 2, and for acceptable results at least 2^1 I think.

Longer windows let us discriminate more frequency bins (1 + half window size). More steps per window should allow finer variations of gain for each band. The tinkles -- which result fom misidentifaction (either way) of a band in a window as noise or not -- would be, if not prevented by other means, then further mitigated by averaging of their effects with overlapping windows where we did not err in identification.

[Paul L]

The methods using frequency domain data are different and the reasons for artifacts may be different.

Yes, they may be.
Has anyone tried "gating" the spectrum less viciously?

Are you suggesting a continuous gain curve instead? This sounds like a third smoothing, besides time and frequency!

[Paul L]

Then there might be applying Hann or other window before FFT, which would be a fourth smoothing...

But I think the "less vicious gating" is a lesser-priority thing to try. Here is why:

Remember that what Noise Removal "sees" is like the spectrogram view of your sound, with 2048 sample rectangular windows -- but with only every 1024th sample time shown.

From these data, very refined in frequency but sparse in time, NR must decide "what amplitude is" for a given window and band, and THEN remap amplitude by some gain curve. For each window and band, the present algorithm is trying to decide from only two times, taking minimum.

So the first problem to fix is avoiding "garbage in" to the gain curve. I think that "garbage in" because of time sparseness is the more serious cause of artifacts.

I think a problem with NR now (and what I'm fixing as discussed in the other thread) is that it is fooled by outlying values, which you can easily see in spectrograms of sounds with wavering pitch with rectangular windows.

(I thought (as experiment confirmed) that taking maximum would eliminate many distortions of foreground sound but introduce them in background, which might then be compensated with increased sensitivity setting without harming foreground. Now I think median of at least three would avoid both problems. Will try soon.)

[Robert J. H.]

I am not sure I understand multi rate FFT or twofold windowing.

Multi-rate works at different sample-rates and prepares the result approbriately--as a wavelet for example.
In our case, it would simply mean that the attack release times or the time smoothing would be different according to frequency.

Using two windows is even simpler.
A simple example:
Rectangular windowing with 50 % overlap:
the window function is something like

Code: Select all

W(n)=0.5

Three frames of six samples each give:

Code: Select all

0.5 0.5 0.5 1 1 1 1 1 1 0.5 0.5 0.5

If we want the windowing on the analyse and synthesis sides, the Princen-Bradley condition must be fulfilled:

Code: Select all

Window_squared(n)+Window_squared(n+M) = 1

Where n is the running sample and M the hop size.
Window_squared = Analyze-Window x Synthesis-Window
This means for the rectangle case that we have to take the square root of the window function, i.e we multiply the signal with 0.7071 instead of 0.5.
The same holds true for other windows that are identical on the A&S side--take the square root of the intended window function.
It is clear that you can choose any window for the analysis part, as long as the synthesis window corrects it such that each sample has unity gain eventually.

[Paul L]

And here's a minor UI nuisance I just found...

7) If you change the attack/decay from the default of 0.15, you can't set it back to that value again exactly. (0.10 gets remembered in preferences.)

Sorry, I was confused, ignore that. BUT replace it with this more important #7:

7) Doing Get Profile in a project at one sample rate, then doing Remove Noise in a project with a different sample rate, does the wrong thing.

I don't yet have a test case but I predict this. The program ought to check for this condition and just refuse to do it, I suppose.

Because the assumption in the code is that the frequency bins of the 2048 sample windows used when gathering the profile information, correspond to the frequency bins for 2048 sample windows in in the sound that is later treated, but they won't.

So if I profile in a project sampled at 44100 Hz, say, then remove noise in a project at 88200 Hz, I predict that it will remove sounds an octave higher than in the profile. This could be tested easily with some pure sine wave as the noise profile and some white noise as the sound to be treated. Where does the spectrum of the white noise get notched?

[Gale Andrews]

"Residual" is a bit like "Wet only" in Reverb isn't it

It's very much like "Residual" in "Hard Limiter" in that it refers to the part of the signal that would be removed.

OK but I think we'll have to find another term than "Residual". It's not (at the moment) planned as a slider, so the aim is to return all that would be removed, and only that.

Gale

[Paul L]

Multi-rate works at different sample-rates and prepares the result approbriately--as a wavelet for example.
In our case, it would simply mean that the attack release times or the time smoothing would be different according to frequency.

I don't understand this.

Using two windows is even simpler.

...

Code: Select all

Window_squared(n)+Window_squared(n+M) = 1

Where n is the running sample and M the hop size.
Window_squared = Analyze-Window x Synthesis-Window
This means for the rectangle case that we have to take the square root of the window function, i.e we multiply the signal with 0.7071 instead of 0.5.
The same holds true for other windows that are identical on the A&S side--take the square root of the intended window function.
It is clear that you can choose any window for the analysis part, as long as the synthesis window corrects it such that each sample has unity gain eventually.

I almost understand that.

The existing effect uses rectangular for analysis and Hann for synthesis which satisfies the condition but I believe gives poor results for removing noise from behind sounds. It would be much better to switch them. In fact that is what happened in very old versions of NoiseRemoval.cpp. At some point this was undone for reasons I don't know. I am trying to put the Hann window back in the analysis. Can you suggest other pairs of window functions that might work better? I thought of using Blackman-Harris for analysis but I haven't figured out what to do for synthesis.

[Robert J. H.]

Using two windows is even simpler.

...

Code: Select all

Window_squared(n)+Window_squared(n+M) = 1

Where n is the running sample and M the hop size.
Window_squared = Analyze-Window x Synthesis-Window
This means for the rectangle case that we have to take the square root of the window function, i.e we multiply the signal with 0.7071 instead of 0.5.
The same holds true for other windows that are identical on the A&S side--take the square root of the intended window function.
It is clear that you can choose any window for the analysis part, as long as the synthesis window corrects it such that each sample has unity gain eventually.

I almost understand that.

The existing effect uses rectangular for analysis and Hann for synthesis which satisfies the condition but I believe gives poor results for removing noise from behind sounds. It would be much better to switch them. In fact that is what happened in very old versions of NoiseRemoval.cpp. At some point this was undone for reasons I don't know. I am trying to put the Hann window back in the analysis. Can you suggest other pairs of window functions that might work better? I thought of using Blackman-Harris for analysis but I haven't figured out what to do for synthesis.

You'll better start out with two times a Hanning window.
Just take the square root of each window sample.
For different windows, add the mismatch to the synthesis window. It would thus be a modified Han function that could even be negative at times (but so does the flat-top also).
I would strongly recommend to use zero-padding as well (which is hard to do with rectangular windows because the zeros have to be 5 times as long as the rectangle itself).
Blackman-Harris is probably a good choice (< than one function period).
The advantage is that time-aliasing is reduced. It is generally recommended when spectra are heavily modified.
Visit e.g. Jos's (Julian O. Smith) homepage for further information on the subject.

[Gale Andrews]

7) Doing Get Profile in a project at one sample rate, then doing Remove Noise in a project with a different sample rate, does the wrong thing.

I don't yet have a test case but I predict this. The program ought to check for this condition and just refuse to do it, I suppose.

Because the assumption in the code is that the frequency bins of the 2048 sample windows used when gathering the profile information, correspond to the frequency bins for 2048 sample windows in in the sound that is later treated, but they won't.

This sounds like an important thing to test. Can you do so? It has implications for loading presaved noise profiles (if we ever do that) and would have had implications when we preloaded the same NRP noise profile file in the now defunct CleanSpeech mode.


Gale

[Paul L]

7) Doing Get Profile in a project at one sample rate, then doing Remove Noise in a project with a different sample rate, does the wrong thing.

I don't yet have a test case but I predict this. The program ought to check for this condition and just refuse to do it, I suppose.

Because the assumption in the code is that the frequency bins of the 2048 sample windows used when gathering the profile information, correspond to the frequency bins for 2048 sample windows in in the sound that is later treated, but they won't.

This sounds like an important thing to test. Can you do so? It has implications for loading presaved noise profiles (if we ever do that) and would have had implications when we preloaded the same NRP noise profile file in the now defunct CleanSpeech mode.


Gale

I did this in 2.0.5:

1. In a new project with 44.1 kHz rate, generate 440 Hz Sine wave, amplitude 0.8, 10 seconds. Select all, do Effects, Noise Removal, Get Noise Profile. (An extreme "noise" for demonstration purposes.)
2. Make new track, generate 10 s of white noise at amplitude 0.8. Select all of that, do Effects, Noise Removal, settings 48, 0, 0, 0, Remove.
3. Do Analyze, Plot Spectrum, Log Frequency. (Window size and type are unimportant). You see a notch centered near 440 Hz. Good.

But now create a new project. Set its rate to 88200. Repeats steps 2 and 3. The notch in the spectrogram centers near 880 Hz. In other words the same noise profile is misinterpreted as being an octave higher than it is.

This confirms that there is a bug. I think it would be simplest just to make the effect detect the difference of rate and fail.