How many other effects follow this patter of Fourier transform, change of coefficients, then inverse transform? NoiseRemoval includes most of the work for that. Perhaps the common part should be abstracted into another subclass of Effect.
Thought now it seems NoiseRemoval doesn’t get the handling of the first and last 1024 samples quite right.
More thought and experiment, and I discovered another possible problem with Noise Removal. Here is a test case:
Use the former extreme example of a loud 11025 Hz sine wave for Noise Profile, and white noise as the sound to process. Use 48 dB of noise reduction and ZERO frequency smoothing.
Look at the result using Analyze, Spectrum and use a rectangular window of 2048 (just as Noise Removal does).
Shouldn’t we see a very narrow notch, with a width of one bin (43 Hz)? We don’t get a notch that narrow.
I think I know the reason:
Look at the sine wave using Spectrogram and 2048 and Rectangular.
See the funny stuff that happens at the edges, where there is the abrupt transition to a padding of zeroes? As the code is written, Noise Removal does not see the first funny boundary when analyzing the sample, but it does see the other one.
Therefore, it attenuates frequencies near 11025 more than it should.
Maybe this test case is unrealistic, but a noise profile with spikes at 60 and 120 Hz ought to be common.
attack/decay times are half as long as they should be. Test case already described. Have fix.
Resulting sound always has a 1024 sample fade-in from 0. Could fix.
Attenuated sounds sometimes rise again in the last 1024 samples, as seen in test cases for (4). Different explanation. Less sure how to fix it.
Highest and lowest frequency bins are always silenced (not merely attenuated). Test cases described. Have fix.
Noise profiles with narrow spectrum do not result in a narrow notch effect with frequency smoothing off. Test case described. Could fix.
Frequency smoothing uses arithmetic averaging of gain factors instead of geometric. Have fix. Do NOT yet have a convincing test case (because problem 5 interferes with demonstrating it). A subtler problem.
Besides all of these, I could clean up some misleading comments written by confused people and I could make a number of minor performance improvements.
Perhaps I should pick only some of these fixes for near term inclusion in the source tree.
I don’t see that here.
I can either type in 0.15, or use the left right cursor keys to move the slider to 0.15. The setting is correctly saved in audacity.cfg as:
Isolate was just broken from the beginning. Not sure whether to fix it or remove it. Is it supposed to be the difference of NR and original? Why bother when you can easy duplicate & invert? Is it supposed to pass noise with gain of 1? Then what the manual says about its purpose is misleading.
I read this in the 2.0.5 manual page for the effect:
Isolate: Select this option to leave just the noise - useful if you want to hear exactly what the Noise Removal effect is removing.
But “exactly what the Noise Removal effect is removing” suggests to me the difference of the waveforms. The difference between the original and the reduced noise is not the same thing as passing the noise with a gain of 1.
As for what the code is doing, somebody was trying to pass the noise with gain of 1 and the rest with gain of 0, but didn’t think through how this interacts with the attack/release setting, with the weird results I wrote about above. And just try it, and you hear too much of the non-noise.
In fact Isolate gives such a false impression, it used to make me unfairly mistrustful of using the Removal.
Make a queue of windows (width 2048, step of 1024) and a corresponding queue of (arrays of) gains (one for each bin). It covers about twice the attack/decay time and always has an odd number of windows.
Repeatedly:
Shift a window of output from the queue using the final gain factors (if the queue is full; and this is where frequency smoothing happens – averaging of the gain factor with the factors computed for other nearby frequencies). Shift a window of input into the queue. Assign the noise reduction factor as the gain for all bands of the new window.
Examine the middle of the queue and decide (for each band) if it’s noise. If noise, do nothing. If not noise, reassign the gain to be 1. Then, calculate the exponential decay of that gain from 1 to the noise reduction factor over decay time; make the gain for the band not less than that decay curve for later windows. Do similar backwards in time for attack.
So, all gains are the reduction factor by default, UNLESS raised because the window is not noise or is near enough to some not-noise to be affected by the attack/decay setting. Okay?
But what should be the algorithm to isolate noise? This was not well thought out. It is assigning gain of 1 to noise and gain of 0 to not-noise in the middle of the queue. HOWEVER, attack/decay is still calculated. The assigning of 0 to not-noise in the middle cancels the effect of the decay calculation, but the not-noise can still be raised from 0, to values between 1 and the noise reduction, in the attack, resulting in that “swelling” I wrote about somewhere above. So, the less the noise reduction, the LOUDER is the non-noise in the mix made by isolate. AND, the frequency smoothing is still done too. The result is a weird and useless mess.
What is supposed to happen? Was it meant just to assign 0’s and 1’s without any of the time or frequency smoothing? Will that produce something useful to hear (though not the same as the difference of the original and noise-reduced sound)? Should it be implemented and documented that attack/decay and frequency smoothing and the noise reduction level all have no effect on Isolate?
Sadly, the people writing the manual rarely receive detailed information from those contributing new code. This is an example of where the documenters seem to have drawn an incorrect conclusion about what the effect does. The manual suggests that the “Isolate” feature is the inverse of “Remove”, but as you say, that is not the case.
With extreme high settings, what should happen is fairly clear.
If “Noise Reduction” is very high, then when removing noise, all of the noise should be removed leaving only the non-noise (ideally).
If “Noise Reduction” is very high, then when isolating noise, all of the non-noise should be removed leaving only the noise.
What is less clear is what should (ideally) happen for lower settings.
If “Noise Reduction” is zero and “Remove” is selected, we should remove no noise. Output = input.
If “Noise Reduction” is zero and “Isolate” is selected, then there are two possible ideal behaviours:
A) If we think of “Isolate” as the inverse of “Remove”, then the result should be silence.
B) If we think of “Isolate” as “reducing non-noise”, then the result should be no change. Output = input.
The 2.0.5 manual suggests case “A”, but the effect behaviour is (almost) case “B”.
As the effect in 2.0.6 will be the same as now, I’ve changed the 2.0.6 manual to say:
Isolate: > Select this option to keep the noise and reduce other sounds in the selection.
That doesn’t mean it shouldn’t be worked on. It might take between now and then to get corrections.
Noise Removal would be pure gold if it worked better than it does now. Also see: Auto Profile. Noise Removal is ripe for improvements including changing the name to Noise Reduction.
I agree with Koz (and others) that it would be a good change just to change the name of Noise Removal to Noise Reduction.
I like stuff that “does what it says on the tin” - and reduction, not removal, is what the effect does - clarity of purpose should be explicit in the syntax nomenclature, otherwise we find ourselves setting unrealistic expectations for users.
Personally I would like to see such a change made before 2.0.6. Surely such a simple change cannot de-stabilize the code? I can’t change the code - but I’ll be happy to spend time fixing the Manual if this change gets made …
In order to make such a change it would be necessary to consult with Vaughan as the Release Manager for 2.0.6
Peter.
P.S. But of course this simple change does not and should not preclude future work on improving the efficacy of this effect.
The truly minimal change would leave a mismatch between names in the code and names in the dialog. Programmers might live with that. Ah, but would it break saved Chain files and preferences? That might make some users a little unhappy during migration.
Who can explain to me the “tinkle bell” problem and the history of attempted solutions? Some discussion here Missing features - Audacity Support
It looks like Dominic varied his solution to it. I wonder too if the sensitivity slider, added by others later, was motivated in part by this problem.
That wiki page says
The first pass of noise removal is done over just noise. For each windowed sample of the sound, we take a Fast Fourier Transform (FFT) and then statistics are tabulated for each frequency band - specifically the maximum level achieved by at least sampling windows in a row, for various values of .
But I believe n is always 2 (at least with 44100 sample rate).
Then frequency-smoothing is applied so that a single frequency is never suppressed or boosted in isolation, followed by time-smoothing so that the gain for each frequency band moves slowly.
Actually the procedure does first time-smoothing, then frequency smoothing. This incorrect description is replicated in comments in NoiseRemoval.cpp.
The tinkly artifacts happen when individual pure tones are near the threshold to be preserved – they are small pieces of the background soundscape that survived the thresholding, perhaps because the background noise is slightly different from the fingerprint or because the main sound has overtones that are imperceptible but that boost them slightly over the threshold.
I think that some of the artifacts I hear, when time and frequency smoothing are set to minimum values, are the opposite: bits of foreground are mistaken for noise and subtracted causing a drop-out in a frequency band.
I tried rewriting Noise Isolation so that it simply assigns gain factors of 1 to noise and 0 to the rest before inverse FFT, with no time or frequency smoothing. In one trial, this isolates something that does a very good job of cancelling noises in the pauses of speech when inverted and mixed with the original. However, hearing it alone, there are plenty of tinkles inside the words.
So while the Audacity noise gating algorithm could perhaps be improved […]
Already I am speculating about one way to remove the tinkles from the words. It might put them back in the pauses. However, a positive sensitivity setting may then offset that effect. Will experiment today.