HowTo: Remove dither noise from silence

[steve]

Background: It is very common (and the default in Audacity) for 16 bit files to be exported with dither noise applied. Depending on the type of dither used, the noise may be added to absolute silence as well as non-silent audio (Triangle and Shaped dither add noise to silence. Rectangle dither does not).

If you have 16 bit WAV/FLAC/AIFF files that have dither noise where it should be silent, you may want to remove the dither noise from the "silent" sections (so that they become true silence). Here's how to do it (losslessly) using Audacity 1.3.12.

• In Audacity "Edit > Preferences > Quality" set the Default Sample Format to "32-bit float" and High Quality Conversion > Dither to "None".
• In "Edit > Preferences > Interface" set the Meter/Waveform dB range to -145 dB.
• Download and install "NoiseGate version 2.4" from here: http://forum.audacityteam.org/viewtopic ... 10#p100910
• Import the 16 bit file into a new Audacity project.
• With the NoiseGate effect, use the Analyse Noise Level function on a "silent" (dither noise only) section of track. Note the reported noise level (ignore the suggested setting) - it should be around -72 dB if shaped dither was used, and around -90 dB if triangle dither was used.
• (Optional - switch the track display to "Waveform (dB)". This will make it a lot easier to see the dither noise disappear).
• Select the entire track, and call up the NoiseGate effect again and use the settings listed below *
• Export the track in a lossless format.

(don't forget to re-enable dither in Preferences).

NoiseGate Settings:
Select Function: -> Gate
Stereo Linking: -> Don't Link Stereo
Apply Low Cut Filter: -> No
Gate Frequencies Above: -> 0.0
Level Reduction: -> -100 dB
Gate Threshold: -> Set this to 1 dB above level displayed by the Noise Analysis step.
(For example, if the noise analysis indicated that the noise level is -90.3 dB, set the Gate Threshold to -89.3 dB.)
Attack/Decay: -> 250

The picture below shows 4 tracks with Waveform dB track display at the end of a fade-out.
Track 1 - the original 32 bit audio.
Track 2 - The audio from Track 1 after being exported as 16 bit WAV (with dither).
Track 3 - After applying the Noise Gate to Track 2.
Track 4 - After re-exporting Track 3 as 16 bit WAV (without dither).

trackpanel000.png (21.3 KiB) Viewed 19190 times

[Gale Andrews]

Maybe a note why this is better than the built-in Noise Removal for this scenario would be useful to some? I presume it's because Noise Removal will produce artifacts, given it will be trying to remove the noise from all the audio, whereas the gate just works on de-amplifying the noise on its own, the level of which is known?



Gale

[steve]

Maybe a note why this is better than the built-in Noise Removal for this scenario would be useful to some? I presume it's because Noise Removal will produce artifacts,

Yes it does. Noise Removal can reduce the dither noise by about 30 dB, which is just enough to push shaped noise below the bottom bit for 16 bit format. However it also causes distortion to the rest of the sound which outweighs the benefits of the procedure.

On the other hand when using the Noise Gate, sound that is above the noise threshold is totally untouched (provided that the "Gate Frequencies Above" setting is at zero).

Another angle on this is that for non-silence we would usually want the dither noise to be present because it reduces the quantization distortion that would otherwise be created when downsampling from 32 bit to 16 bit. For audio above the noise threshold the Noise Gate leaves the dither noise alone, whereas the Noise Removal effect is attempting to remove it.

The importance of turning dither off while we do this is a) we want to avoid putting dither back into the silence on export, and b) the audio has already been dithered so we don't really want to dither it again.

[steve]

Update - Setting the Attack/Decay to 10 ms is probably better, provided that the noise threshold has been set correctly.

[Gale Andrews]

Another angle on this is that for non-silence we would usually want the dither noise to be present because it reduces the quantization distortion that would otherwise be created when downsampling from 32 bit to 16 bit. For audio above the noise threshold the Noise Gate leaves the dither noise alone, whereas the Noise Removal effect is attempting to remove it.

Funnily enough the majority of people who report dither noise as a "bug" (which it currently is if you are exporting to 16-bit from a 16-bit project) claim that the audio (not just the "silence") sounds better with the dither turned off. Probably that's "hearing what you want to hear", but it does suggest that not everyone hears significant improvements with dithering.


Gale

[steve]

Funnily enough the majority of people who report dither noise as a "bug" (which it currently is if you are exporting to 16-bit from a 16-bit project) claim that the audio (not just the "silence") sounds better with the dither turned off.

If you do a straight import -> Export of 16 bit data, then theoretically the output should be more "accurate" if NO dither is used. In practical terms I'm not sure that I can hear any difference - perhaps a very slight qualitative "difference" at the top end, during very quiet passages, when listening through good headphones, but not really enough to say that one is "better".

Where I find that it does make a significant difference is in periods of absolute silence if the file has been edited on several different occasions (import -> Export -> import -> Export -> import -> Export -> import -> Export). In this case it has dither applied on each export and the noise adds up to a point where it becomes distinctly noticeable. In particular any absolute silences in the original are clearly no longer silent (they have a high frequency hiss).

Repeat the import -> Export enough times and the hiss also becomes noticeable when there is sound (which the noise gate can do nothing to improve). Personally I find that with just one or two applications of dither the noise is almost completely masked by the music in non-silent sections with just a tiny bit of noise present in totally silent sections.

Probably that's "hearing what you want to hear", but it does suggest that not everyone hears significant improvements with dithering.

The thing that I notice most with non-dithered audio (and this only occurs if some sort of processing has been done at a higher bit depth) is that during the last few moments of a fade-out to silence I can hear the sound jump between sample values. I think there is something fancy going on in my sound card because I would have expected this to sound "crackly" with overtones, but surprisingly it just sounds like the original sound is coming and going. When dither is used the decay is smoother and sounds more natural (though with a very low level hiss).

[steve]

This may be better as a new topic, so I've made it as a new post so that it can be easily split-off from the topic.

Experiments with dither.

The attached file has 4 copies of the same sound (a modulated 1kHz sine wave at a very low level fading out).
tremwav.wav is the original sound exported at 32 bit.
tremwav16-no-d.wav is the same sound exported at 16 bit with no dither.
tremwav16-triangle.wav is the same sound exported at 16 bit with triangle dither.
tremwav16-shaped.wav is the same sound exported at 16 bit with shaped dither.

If you listen to each of them in turn they should all sound very similar (and very quiet), illustrating how subtle the effect of dither is.
If you are using a 16 bit sound card the sound will disappear at about 9 seconds as there are no "bits" to represent the final second.

Making the differences more obvious.
Import one of the files into a new project and make a further 3 duplicate copies (select > Ctrl+D > Ctrl+D)
Now play and listen.

The 32 bit version sounds almost identical to the original - just louder, and the fade continues to 10 seconds (the end of the track)
The 16 bit "no-dither" version has distinct harmonic artefacts and there is a slight judder before it stops at 9 seconds.
The 16 bit "with dither" version do not exhibit the harmonic artefacts, but have some background hiss. They also continue to the full 10 seconds.

I don't hear much difference between triangle and shaped dither, but the shaped dither has a slightly higher peak level.

[steve]

Another user case for removing dither from silence.

If you want to use the "Detach at Silences" function, dithered "silence" will not be recognised as silence.
"Detach at Silences" does not have a GUI (it's in the Edit menu) and there is no way to configure a threshold for "silence".
This limitation can be worked round by gating the dither noise to absolute silence.

(this user case was the original inspiration for adding the ability to gate to absolute silence:

This is "almost" exactly what I need, but I need a version modified to clamp to perfect silence, instead of just really really quiet, so that the "Detach at Silences" (a/k/a Disjoin in older versions) function will cause the quiet regions to go away.

http://forum.audacityteam.org/viewtopic ... 878#p93878 )

[steve]

If anyone wants to try using the "HowTo: Remove dither noise from silence" technique with "tremwav" samples (Experiments with dither), I would suggest the following settings for use with tremwav16-triangle.wav

Gate frequencies above = 1.0 kHz
Level Reduction = -100 dB (silence)
Gate Threshold = -80 dB
Attack/Decay = 8000 (must use text input for values over 1000)

Other settings could be used, but the issue to be cautious about is making the sound cut in/out/in/out during the fade. A slow Attack/Decay helps to prevent that from occurring.

[Gordon Coates]

Funnily enough the majority of people who report dither noise as a "bug" (which it currently is if you are exporting to 16-bit from a 16-bit project) claim that the audio (not just the "silence") sounds better with the dither turned off. Probably that's "hearing what you want to hear", but it does suggest that not everyone hears significant improvements with dithering.

One example I experienced of audio sounding better without dither on export was when I opened a 24 bit 192 khz transfer of a 50 or 60 year old analogue tape recording (purchased from "High Definition Tape Transfers") in 32 bit float just to add 30 seconds of silence at the end of the track. Exporting at the original resolution with the default dither settings produced a sound noticeably lacking in the excellent transient attack and rhythmic drive present in the original file, but exporting with no dither solved this problem. One confounding factor might have been the presence of tape hiss in the original recording, and another might have been an unknown amount and type of dither (or perhaps none at all, in view of the existing tape hiss and the high resolution) added during its original digitization.

Since then, I have used a chain to apply an equalization curve (about 11dB cut at 85Hz, rising steeply to no effect below 60Hz and no effect above about 125Hz, but preceded by -2dB amplification because the curve, which showed no boost, was nevertheless occasionally adding up to 1.7dB somewhere or other, causing occasional clipping) to thousands of 16 bit and 24 bit files (in order to optimize them for replay via modified Quad ESL57 loudspeakers on non-standard supports placed fairly close to room boundaries) always using no dither on export, and I haven't yet noticed any loss of sound quality in any of the exported files. (However, I haven't tried any fade-outs!)