Noise Removal Tests

Effects, Recipes, Interfacing with other software, etc.

If you require help using Audacity, or think you have found a "bug", please post on the forum board relevant to your operating system.
Windows
Mac OS X
GNU/Linux and Unix-like

Noise Removal Tests

Permanent link to this post Posted by billw58 » Sat Sep 11, 2010 4:21 pm

Reading this thread: viewtopic.php?f=28&t=7010 led me to investigate the NR effect in 1.3.13.

My method was as follows:
    New Project
    Generate noise, white, level 0.01, 10 seconds
    New track
    Generate tone, sine, 400 Hz, 9 seconds
    Move tone track so the two tracks end at the same time
    Select All then Mix and Render
    Export to WAV or AIF
    Select noise only portion, Effect > Noise Removal and Get Noise Sample
    Select entire track
    Effect > Noise Removal with Noise Reduction 12, Sensitivity 0, Smoothing 130 and Attack/decay 0.10

Result:
    Noise is removed from the first second of the track
    Once the tone starts, noise is not removed, and instead "wobbles" in volume

In my understanding, this is not correct behaviour. If the NR algorithm is using FFT to sample the noise profile, then applying downward expansion to each frequency band according to the level of audio in that band, the noise should be removed even in the presence of the tone. Noise would remain the the band containing the 400 Hz tone, but would be removed in all other bands. Thus the "masking" effect would make it appear that the noise has been completely removed.

To confirm this:
    Open the exported noise+tone file in Brian Davies' DeNoise program
    Set "Limit reduction to 12 dB"
    Check the "Automatic" checkbox
    Click the "Start" button

Import the DeNoised file in the same project and compare the two results.

The DeNoised file has the noise removed in the presence of the tone.

Virtually identical results can be achieved in DeNoise by setting the "White" checkbox, or sampling the noise profile at the start of the file per instructions on page 14 in the DeNoise manual.

Possible conclusions:
1) DeNoise is superior to Audacity's Noise Removal effect.
2) The Noise Removal effect is broken.

Conclusion #1 seems obvious from this experiment.

Conclusion #2 is speculative. It all depends of how the Noise Removal effect is intended to operate. If the NR effect is intended to operate as described above, then I submit that it is broken.

-- Bill
billw58
Forum Staff
 
Posts: 3167
Joined: Wed Aug 12, 2009 2:10 am
Location: Waterloo, Ontario

Re: Noise Removal Tests

Permanent link to this post Posted by waxcylinder » Sat Sep 11, 2010 4:39 pm

Interesting Bill - this may have some bearing on some recent posts I can vaguely recall where folks had problems with Audacity's NR.

Are you planning to potentially report this as a bug?

WC
________________________________________FOR INSTANT HELP: (Click on Link below)
* * * * * FAQ * * * * * Tutorials * * * * * Audacity Manual * * * * * Audacity Wiki * * * * *Donate * * * * *
waxcylinder
Forum Staff
 
Posts: 7390
Joined: Tue Jul 31, 2007 11:03 am
Location: Manchester, UK

Re: Noise Removal Tests

Permanent link to this post Posted by billw58 » Sat Sep 11, 2010 6:13 pm

waxcylinder wrote:Are you planning to potentially report this as a bug?


Potentially, yes. I'm hoping to get some indication from those in the know of how the effect is meant to operate, and if my test is valid.

-- Bill
billw58
Forum Staff
 
Posts: 3167
Joined: Wed Aug 12, 2009 2:10 am
Location: Waterloo, Ontario

Re: Noise Removal Tests

Permanent link to this post Posted by billw58 » Sun Sep 12, 2010 4:03 pm

Further tests ... Gale and Steve suggested I try increasing the Sensitivity slider to get more aggressive noise reduction. The results were not much different.

Plot Spectrum of the unprocessed tone+noise:
SpectrumNRbefore.png
SpectrumNRbefore.png (74.79 KiB) Viewed 9410 times


Plot Spectrum after Audacity NR with Sensitivity = 0:
SpectrumNRafter.png
SpectrumNRafter.png (77.22 KiB) Viewed 9411 times

As you can see, the noise that is spread out along the -78 dB line is virtually untouched. There is some minimal reduction above 6000 Hz. The increase below 40 Hz is the "wobble".

Plot Spectrum after Audacity NR with Sensitivity = 12
SpectrumNRplusSensAfter.png
SpectrumNRplusSensAfter.png (78.64 KiB) Viewed 9410 times

There is now some clear NR above 3000 Hz. The wobble remains.

Plot Spectrum after processing by DeNoise:
SpectrumNRafterDavies.png
SpectrumNRafterDavies.png (77.55 KiB) Viewed 9411 times

There is now clearly 12 dB of NR up to about 10 kHz. No wobble.

More off-forum discussion has occurred while I was composing this. Might as well post it, then respond on -quality to the comments. I'm posting here instead of -quality in order to include the attachments. Hope that's OK.

-- Bill
billw58
Forum Staff
 
Posts: 3167
Joined: Wed Aug 12, 2009 2:10 am
Location: Waterloo, Ontario

Re: Noise Removal Tests

Permanent link to this post Posted by steve » Sun Sep 12, 2010 6:26 pm

There is a pretty clear bug/unintended behaviour showing up in this test.
Irrespective of what settings are used, the Noise Removal effect generates low frequencies (as shown by the peaks at around 12 Hz and 30 Hz in the screen shots).
This low frequency noise is audible.

Apart from that problem, pretty good results can be obtained using these settings:
Noise Reduction = 24
Sensitivity = 18
Frequency smoothing = 100
Attack/Decay = 0.0

screenshot.png
screenshot.png (38.98 KiB) Viewed 9409 times


With the following settings, the plot looks almost identical, but when listening the noise has been almost completely eliminated.
NR = 48
S = 16
FS = 100
A/D = 0.0

screenshot2.png
screenshot2.png (39.05 KiB) Viewed 9407 times


Listening to the "isolated" noise reveals that virtually nothing has been removed from the original sine tone, but the low frequency noise is also clearly evident.
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)
steve
Senior Forum Staff
 
Posts: 34430
Joined: Sat Dec 01, 2007 11:43 am

Re: Noise Removal Tests

Permanent link to this post Posted by steve » Sun Sep 12, 2010 6:41 pm

BTW, the current version of Audacity NR (with "Sensitivity" adjustment) is a considerable improvement over the previous version (currently in Audacity 1.3.12).

There is also another bug in Noise Removal (1.3.13 alpha):
The Sensitivity and Attack/Decay sliders do not respond correctly to keyboard control. (tested on Linux only)
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)
steve
Senior Forum Staff
 
Posts: 34430
Joined: Sat Dec 01, 2007 11:43 am

Re: Noise Removal Tests

Permanent link to this post Posted by billw58 » Mon Sep 13, 2010 1:24 am

stevethefiddle wrote:There is a pretty clear bug/unintended behaviour showing up in this test.
Irrespective of what settings are used, the Noise Removal effect generates low frequencies (as shown by the peaks at around 12 Hz and 30 Hz in the screen shots).
This low frequency noise is audible.

Apart from that problem, pretty good results can be obtained using these settings:
Noise Reduction = 24
Sensitivity = 18
Frequency smoothing = 100
Attack/Decay = 0.0


Now try those settings on a real-world example. For instance, import a CD track, add white noise at -40 dB and apply those settings. The result is horrible.

-- Bill
billw58
Forum Staff
 
Posts: 3167
Joined: Wed Aug 12, 2009 2:10 am
Location: Waterloo, Ontario

Re: Noise Removal Tests

Permanent link to this post Posted by steve » Mon Sep 13, 2010 2:29 am

billw58 wrote:Now try those settings on a real-world example. For instance, import a CD track, add white noise at -40 dB and apply those settings. The result is horrible.

Of course the results are horrible :D but I think that is half of the point - the optimum settings are dependent on the type of audio and the type of noise. If there were settings that worked regardless of the material, then the effect could be preprogrammed with the optimum values and no sliders. To compare the performance of different Noise Removal effects you need to specify the test material, then compare the optimum settings for that test material (which is much harder to do with real world samples). I don't think that Audacity NR is particularly good at removing white noise, but does it do better with other types of noise? Many specialist audio restoration programs will have several different effects for dealing with different types of noise (hum, buzz, hiss, crackle, pops...). Other than "Click Removal" Audacity has just one NR.

Artificial tests can indicate possible issues (such as the "wobble" as you called it/"low frequency noise" as I called it) but are quite limited when it comes to assessing what real world performance will be like. As an example, the two plots that I posted look extremely similar, and I think that you would agree that you would expect them to sound virtually the same, but if you repeat the test you should notice that they sound remarkably different.

I'm just wondering what sort of tests we can manage -
Do you have a licensed version of DeNoise? (I don't, but I could run the 21 day trial)
What are your C++ skills like? (mine are non-existent)
Do you only run Macs or other platforms as well? (I'm mostly on Linux, but have XP in VirtualBox)
Do you have any other Noise Removal programs for comparison? (I've got Gnome Wave Cleaner)
Do you have a host program that fully supports VST effects? (I don't, but I can get limited access to one).
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)
steve
Senior Forum Staff
 
Posts: 34430
Joined: Sat Dec 01, 2007 11:43 am

Re: Noise Removal Tests

Permanent link to this post Posted by billw58 » Mon Sep 13, 2010 2:55 am

stevethefiddle wrote:I'm just wondering what sort of tests we can manage -
Do you have a licensed version of DeNoise? (I don't, but I could run the 21 day trial)
What are your C++ skills like? (mine are non-existent)
Do you only run Macs or other platforms as well? (I'm mostly on Linux, but have XP in VirtualBox)
Do you have any other Noise Removal programs for comparison? (I've got Gnome Wave Cleaner)
Do you have a host program that fully supports VST effects? (I don't, but I can get limited access to one).


Yes I have a licensed version of DeNoise.

I have zero C++ skills :(

Mac only, I'm afraid, so can't try GWC.

No programs other than DeNoise. I could download the 14-day trial of SoundSoap SE, but it apparently mangles the output during the trial period.

And no programs other than Audacity that support VST effects (and my Mac is prone to the "one VST effect or I'll hang" bug).

I've been thinking of real-world sources that would expose what I see as the shortcomings of the effect. A cappella vocals, solo guitar, and just about any classical piece (lots of dynamics)? Perhaps adding noise at -40 dB is too severe. Perhaps -50 or -60 would be more realistic, but at -40 it's easy to hear.

There's a parallel discussion on the -quality list (as you know), but I'll repeat something here for those who are not subscribed to that list. It appears that the Audacity NR effect is not doing multiband noise gating [I'll call it "gating" although it is really downward expansion] with the gate thresholds set by the noise profile. Instead it seems to be using the overall full-spectrum signal level to decide whether or not to open or close the gates. The upshot is that as soon as the signal starts, the noise removal is negligible to non-existent. Raising the Sensitivity slider just causes the effect to treat more of the signal as noise.

-- Bill
billw58
Forum Staff
 
Posts: 3167
Joined: Wed Aug 12, 2009 2:10 am
Location: Waterloo, Ontario

Re: Noise Removal Tests

Permanent link to this post Posted by Gale Andrews » Mon Sep 13, 2010 7:38 am

Here are some more comparisons in 1.3.13 HEAD with Bill's tone/noise sample showing the spectrum of the tone after processing (Noise reduction 34 dB, FS 140 Hz, A/D 0.00):

First with Sensitivity at 0.00 dB:
1.3.13_hiss_not_removed.png
1.3.13_hiss_not_removed.png (877.54 KiB) Viewed 9394 times

Little hiss suppression evident or audible.

Second with Sensitivity at 20.00 dB:
1.3.13_hiss_removed.png
1.3.13_hiss_removed.png (877.54 KiB) Viewed 9390 times

This seems to show the hiss reduced by about 30 dB above 3000 Hz. Subjectively I can't hear the original hiss at all.

Then a comparison with 1.2.6 with the slider half-way (using the 1.3.13 Plot Spectrum of course):
1.2.6_processed_noise.png
Bill's tone/noise sample processed by 1.2.6
1.2.6_processed_noise.png (877.54 KiB) Viewed 9390 times

This seems to show little hiss reduction on the face of it. There are no low frequency spikes comparable to 1.3.13 but there is a significant spike at 300 Hz. Here is a wav from that processed 1.2.6 audio:
126noise.wav
(430.72 KiB) Downloaded 219 times

To me I am not audibly sure if much hiss has been removed in 1.2.6 or not. There is less wobble than in 1.3.13 but the main thing I hear is the 300 Hz which now makes it sound like a dual tone. The 300 Hz does seem to smother the hiss but I still sense there actually is less hiss and that the -78 dB audio sounds like a mix of hiss and quiet tone artefacts.

My conclusion, none of the results are perfect in view of the fairly loud artefacts to the tone. 1.2.6 and 1.3.13 both produce tone artefacts at about -42 dB, but at different frequencies. The 1.3.13 "hiss removed" audio sounds much the best to me.

How does the DeNoise tone sound after processing? Is it much less artefacted? I agree we should be worrying about real life samples more than tone artefacts.

Note that Marco's extra code (in the Windows Unicode Nightly) gives a better result again in my view (tone artefacts less audible, although the plot looks similar). However you have to sample the noise and the signal with that or the results are often worse than in SVN HEAD (more artefacts, and large amplitude reductions in the result).


Gale
________________________________________FOR INSTANT HELP: (Click on Link below)
* * * * * Tips * * * * * Tutorials * * * * * Quick Start Guide * * * * * Audacity Manual
Gale Andrews
Quality Assurance
 
Posts: 14113
Joined: Fri Jul 27, 2007 12:02 am

Next

Return to Audio Processing



Who is online

Users browsing this forum: Google [Bot] and 5 guests