Analyzing DC Offset

[Edgar]

The Normalize effect is supposed to analyze and remove DC offset at the users request; in my recent tests on 2.0 and newer Audacitys this does not seem to be working properly. The Noise Removal effect is not advertised as doing this analysis and removal but it does.

Quite a few months back there was some talk about splitting out the code for removing DC offset from Normalize and making it a separate effect of its own. My recollection is that the Developers decided that it was not worth adding an extra menu item to the Effects menu since everyone was already accustomed to using the Normalize effect to perform this operation.

[kozikowski]

everyone was already accustomed to using the Normalize effect to perform this operation.

Yes. Everybody who already knows where the tools, knows where the tool is. Does it show up in a search for DC Removal? It's not in the Index.

Koz

[steve]

Topic split from http://forum.audacityteam.org/viewtopic ... 64#p203151

The Normalize effect is supposed to analyze and remove DC offset at the users request; in my recent tests on 2.0 and newer Audacitys this does not seem to be working properly.

What's the problem? Can you give steps to reproduce the issue?

The Noise Removal effect is not advertised as doing this analysis and removal but it does.

The Noise Removal effect has a 6 dB/octave high-pass filter at about 30 Hz. I don't think that is in the documentation, but I don't recall anyone ever asking about it before.

Quite a few months back there was some talk about splitting out the code for removing DC offset from Normalize and making it a separate effect of its own.

That discussion has gone on a lot longer than "a few months"
I'm in favour of a separate DC offset tool, though the lack of Effects management is a big stumbling block for the addition of any new effects.

Yes. Everybody who already knows where the tools, knows where the tool is.

Absolutely

[kozikowski]

Programmers love that recursive logic.

"If you can't read this, Call Operations x76....."

We actually had that message for a short time. I'm not making that up.

From the legacy discussion of this, DC Removal would get you into trouble by applying any filters at all. So what it does is average out the whole show and apply a compensating DC voltage if needed. Remember that?

Nobody could tell me a **real-world** instance where that wouldn't work, but we could generate very serious instances of filters failing.

The upshot was leave the averaging tool right where it was.

Koz

[steve]

Nobody could tell me a **real-world** instance where that wouldn't work

I've had similar doubts, but I eventually found one: http://forum.audacityteam.org/download/file.php?id=6434
See what happens if you apply DC offset correction (with the Normalize effect) to this sample.

[Edgar]

The Normalize effect is supposed to analyze and remove DC offset at the users request; in my recent tests on 2.0 and newer Audacitys this does not seem to be working properly.

What's the problem? Can you give steps to reproduce the issue?

Hi Steve!
The following is the exact text from an email I sent you on 24 December and also sent Gale on 7 January; given the season and the push to get 2.0.3 out I was not too surprised to never hear back:

I am having a hard time understanding what is going on here… Maybe if we start with a concrete example:
http://www.youtube.com/watch?v=wfxv4jcGBkI
THE WHISTLING COWBOY ~ Horace Heidt & his Orchestra 1942

First, on Audacity's Preferences page Import/Export in the "When importing audio files" section, make sure that the check box for Normalize all tracks in project" is turned OFF.

Download a copy of this video and import it into Audacity. Just for the sake of reference, make a region label from .02 seconds through .2 seconds, call this "on". Make a second region label from .1 seconds through .15 seconds, call this "noise". Create a final region label from 61 seconds through 62 seconds, call this "above".

Remember, this file has just been imported and not Normalized.

Zoom in closely to the "on" section, a close inspection leads one to believe that there is little or no DC Offset; pay attention to the "centeredness", which looks good, up through the entire first section of about one second or more.

Zoom in closely to the "above" section. Visual inspection leads me to believe that there is DC Offset (see the attached above.PNG). Now, select the entire track and Normalize it (making sure that the check box for "remove DC Offset" is checked). Afterwards, a visual inspection of the "above" section leads me to believe that the audio is now centered. Go back to the beginning of the track at the section labeled "on" and zoom in closely – it's visually apparent that this previously OK audio has been shifted in the negative direction by quite a bit (even out past the one second mark).

Select the audio represented by the "noise" region (even at the sample zoom level it appears to be absolute silence), select the Noise Removal effect and use this selected audio as the profile (Get Noise Profile); select the entire track and remove the noise profile. If you are not still zoomed in closely to the "on" section do so and visually inspected – it appears now to be happily back at zero – even out past the one second mark.

Zoom in on the "above" section again; visually, it is still centered.

It seems that I must always run a second effect – Noise Removal, Click Repair, High Pass Filter etc. it doesn't seem to matter which – after I do a Normalize to repair "damage" caused by Normalize. I suspect this is very subtle and I have not dug out a set of headphones to do a A/B comparison of all the possible combinations. What do you think, does this bear closer inspection?

above.png (187.91 KiB) Viewed 5172 times

The above was the extent of my email to Steve and Gale. Note that after further experimentation I realize that the final paragraph is incorrect in that Noise Removal is the only effect that I know for sure resolves the DC Offset.

[steve]

from an email I sent you on 24 December

Sorry Edgar - I was off the radar for a few days over Christmas and missed it.

What has happened there is that someone has taken a recording that has significant (positive) DC offset, faded out the ends, and added a bit of silence to the ends. When the offset is corrected, it is "pulling down" the start and end of the track below zero - in effect, introducing a localised offset at each end.

The limitation of DC offset correction (as implemented in the Audacity Normalize effect) is that it looks literally and only for DC - that is, an offset that has zero frequency.
Another way to define DC offset is "the average of all samples in the selection".

In the case of the whistling cowboy, there is zero offset at the start, zero offset at the end, but an offset of about 0.07 (linear) throughout the rest of the track.

Here is a handy little snippet of Nyquist code that will display the amount of offset as a rolling average over 1/4 second intervals.

Code: Select all

(defun bias-detect (sig)
  (let ((step (round (/ *sound-srate* 4)))) ; 1/4 second in samples
    (force-srate *sound-srate*
      (snd-avg sig step  step op-average))))

(multichan-expand #'bias-detect s)

If you apply that code (in the Nyquist Prompt) to the whistling cowboy, you can see clearly what has happened.
Note that this "offset" is not strictly "DC offset" because it is not a constant voltage. For this reason I prefer the term "bias", which can refer to DC bias, or to a slowly changing offset.

Noise Removal is the only effect that I know for sure resolves the DC Offset.

Which is why I invented "dynamic bias adjustment", as used in the dc-offset tool: http://forum.audacityteam.org/download/file.php?id=6436
and is why I intend to campaign for a better tool in Audacity for dealing with offset (after 2.0.3 has been released).

[Edgar]

Steve, thanks for the detailed info and the tools. I wondered if it was (my) operator error--in this case a failure to understand the details <grin>!

[steve]

It's interesting to apply that "rolling average" code to the "trumpet" audio clip in this post http://forum.audacityteam.org/viewtopic ... 92#p203192

[Edgar]

I need some documentation for that new DC offset removal tool.

The first control "Apply or remove…" would seem obvious until you see the third option "remove then add". It also begs the question "why would I add offset" unless the implication is that "if the offset is above the zero line – remove it; if the offset is below the zero line add some to get rid of it".

Some clue as to what each removal method does and the appropriate circumstance under which one would use that type would be beneficial. This also begs the question what method is used if one adds offset.

Finally there is the "How much to add…" control which looks like it is a ratio– this brings up at least two questions – do I need to specify an amount if I am removing offset; and, what does the ratio mean – what is it a ratio of?

From the looks of the dialog it looks like this tool is trying to provide two similar tools – a removal tool which analyzes the audio and centers it on the zero line; and, some other kind of tool which is designed to move the audio's average center up or down. I can see that the code would be highly related but mixing the two interfaces (GUI) is confusing. If this effect is in fact two tools then I would break the interface up into two sections a "DC Removal" section and an "Audio __xxx__" section (fill in the blank with appropriate wording).