ANDing / XORing two almost identical tracks

Special Interest Groups Audio Processing
1

I use Audacity (1.3.5, WinXP, SP2) to record various sounds, eg, from YouTube, the TV card, microphone, etc., using “What U Hear”. The sources are all mono, but by default I record in stereo, then split and save as a mono mp3.

Here’s the problem: past a certain CPU loading on my PC, my soundcard starts sending clicks to the speakers, and Audacity captures them. On average, there’s a click every 3 seconds or so, but there can be a burst of a dozen clicks in a couple of seconds. For most recordings, there are too many to consider manual removal. Now, this isn’t a show-stopper, because the recordings are for convenience rather than for critical use. Still, I’d rather have the clicks gone, as long as the effort is minor.

A possible way to attack the problem mechanically: examining the waveforms in Audacity, I noticed that the clicks rarely occur in both channels at the same time, and that they occur primarily on the negative (lower) side of the wave in the left channel, and the positive side in the right channel. IOW, the “click” noises are exposed as differences in the two waveforms. I’ve uploaded a screen cap with differences – the clicks – highlighted.

Finally, the question: is there a method or a plug-in which can “AND” the two tracks, yielding an output track which excludes any part of either input track’s waveform that is not overlapped by the other input track’s waveform? or, vice-versa, which can “XOR” the two tracks, yielding an output track which includes only non-overlapping parts of the waveforms, ie, virtually all of the clicks, which could then be inverted and merged back into a combined mono version of the source tracks?

Or am I dreaming?
WaveformDifferences.jpg

2

Unfortunately the logic does not wok out - What you are asking for is basically the same as “vocal isolation” as opposed to “vocal removal” (something that is frequently requested, but the logic does not work out). "And"ing the tracks will give yo a mono track that includes 2 x the music + clicks from both channels. XOR will give you a mono track containing the clicks from both channels (like the vocal removal effect), but inverting it and adding to the original tracks will give you the music + inverted clicks from the other channel. There is no way to cancel out the clicks.

Let’s approach the problem from a different direction.

Switching the default recording settings to “1 channel (mono)” will reduce CPU load and should therefore eliminate the clicks.

3

Hi, Steve. Thanks for the quick response.

As regards approaching the problem from a different direction, I can say with confidence that recording in stereo doesn’t cause the clicks; the clicks are happening even before I launch Audacity. I upgraded sound card drivers thinking that might help, got saddled with worse problems, reverted to install CD. Reboot to clear out running processes? miss 2 minutes of live program. New system? wallet anorexic. More excuses available on request.

I don’t have any practical knowledge about digital sound files, but let me try to persuade you that this is a different problem from either vocal isolation or vocal removal.

What we have here is a signal which is true mono recorded on two channels, but noise which appears to have stereo characteristics, and each individual noise incident is limited to one channel only. Check out the attachment - left and right are identical to a very high degree, and the only noticeable deviation is the click in the right channel. So perhaps we can tackle this by hunting down and eliminating these deviations. Whenever there is a difference, we eliminate the sample which is farthest from zero.

My speculation is based on a stereo digital sound file being composed of two tracks with thousands of time-aligned samples per second, each sample with an amplitude ranging from let’s say 255 to -255. Then the filter might work like this (line numbers and “goto” courtesy of my early exposure to BASIC):

010 SampleA = value of first sample from input track A
020 SampleB = value of first sample from input track B
030 AbsValA = absolute value of SampleA
040 AbsValB = absolute value of SampleB
050 if AbsValA > AbsValB
060    write SampleB to output track C
070 else
080    write SampleA to output track C
090 endif
100 SampleA = value of next sample from track A
110 SampleB = value of next sample from track B
120 goto 030

No error trapping, but the basic logic seems sound to me. This process doesn’t sum the two waveforms, it performs something analogous to a bit-wise AND, in the sense that the lower value “wins”. If I’m out to lunch here, I’d appreciate it if you could point out where my logic derails.

Thanks in advance, Steve.
WaveformZoom.jpg

4

What steps have you take to reduce the load on your CPU? Audacity wakes up from first birthday in mono. What happens if you leave it that way?

The other troubleshooting trick is to break it really good. What can you do to make the clicks much worse?

Have you run a Memory Checker? Microsoft makes a terrific one for free. If you can’t remember how to make a bootable floppy, there are instructions to make a bootable CD.

http://oca.microsoft.com/en/windiag.asp

We can’t assume anything, so you have error checked and defragged the hard drive, right, and there’s at least 20% free space? Are you still running the printer manager in high memory for that printer you don’t own any more?

I can do passable production (slowly) on a 233 MHz Windows 98SE machine, and real work on a 733 MHz Win2K machine, so your machine isn’t overloaded. It’s broken or badly misadjusted.

Koz

5

In defence of my suggestions, that seems to contradict what you said in the first post: “past a certain CPU loading on my PC, my soundcard starts sending clicks to the speakers, and Audacity captures them”
Your earlier statement clearly suggests that the solution is to reduce CPU loading.

As Koz points out, there are several places that “data bottlenecks” can occur and cause these kinds of problems, and it would still be better to fix the cause of the problem than to try and deal with damage to the audio that results.

There are a load of suggestions for improving audio playback / recording performance in the “Tips” section of the Audacity wiki: http://audacityteam.org/wiki/index.php?title=Main_Page

Common causes of this kind of problem include:
Lack of disk space / disk fragmentation
Shortage of memory (real RAM memory - not virtual memory)
Anti-virus programs
Windows update
Viruses and spyware
Other programs and processes running in the background
USB devices are particularly susceptible to these types of problem, and there is a section on the wiki that specifically deals with USB.

Back to the “AND / XOR” question.
If dealing with the tracks as data streams, then the problem is the same as vocal isolation, but as you point out, in this special case we may be able to deal with it by bit-wise processing.

The original screen shot was not zoomed in enough to tell, but if as you suggest (and appears in the detail picture), the clicks are always samples that are greater (absolute value, disregarding sign) than the correct audio, then yes, you could correct the problem by bit-wise processing. To do this, you would write a Nyquist script; http://audacityteam.org/wiki/index.php?title=Nyquist_Basics:_The_Audacity_Nyquist_Prompt

Before you get too excited, there are a couple of practical problems/issues:

  1. Are the clicks always greater sample values than correct audio signal?
  2. Bit-wise processing can be very slow. The processing will probably be considerably slower than real time.
  3. Bit-wise processing is very memory heavy. If the processing does not all occur in RAM, then the speed will slow down even more, and some processes must use RAM.

Here is an example of Nyquist code that will do this. Note that the maximum length is set to 1000000 samples.
Also note that any clicks that have absolute values that are less than the sample value of the “correct” sound will be assumed to be correct and will be written to both channels.

(setq maxlen 1000000) ;; maximum length that can be processed (samples)

(setq samplerate (snd-srate (aref s 0))) ;; find sample rate
(setf common (make-array (snd-length (aref s 0) maxlen)));; initialise array

;; The complicated bit - this is the start of a "do" loop that
;; counts up from 0 until it runs out of samples
;; it also sets "left" to read samples sequentially
;; from the left audio channel, and "right" to read samples from the right channel

(do ((n 0 (1+ n)) (left (snd-fetch (aref s 0)) (setq left (snd-fetch (aref s 0))))(right (snd-fetch (aref s 1)) (setq right (snd-fetch (aref s 1)))))

;; Exit when we run out of samples (v is nil) and return the "common"array
((not left) common)

;; Start the execution part of the "do" loop

;; write the sample with the lowest absolute value to the array
(if (> (abs left)(abs right))(setf (aref common n) right)(setf (aref common n) left))

) ;; End of "do" loop

;; convert back to sound
(vector (snd-from-array 0 samplerate common) (snd-from-array 0 samplerate common))
6

Oops, I nearly forgot - how to use that code.

Select some audio - it must be a 2 channel track, and no more than a million samples in length.
Select “Nyquist Prompt” from the Effects menu.
Copy and paste that code into the Nyquist prompt text box.
Click “OK”

If you want to process more than 1 million samples at a time, change the first line in the code to indicate the maximum length (in samples).

(setq maxlen 1000000)

If you decide to increase the number, keep an eye on how much memory it is using - if it runs out of available memory Audacity will crash. With a modern machine you should be able to go up to 100 million samples or more without any problem.

7

And through all that, what you get at the end is a patched sound track, not the clear one you would have by fixing your machine.

Koz

8

Yes you are absolutely right Koz, it’s much better to avoid clipping in the first place than to try and repair it after the damage has been done. The same can be said of noise reduction, click removal, and all other audio restoration, but sometimes the option of re-recording is not an option.

ClipFix works best when it is just called upon to repair the occasional peak that has been allowed to exceed 0dB rather than to try and repair a complete track that has been “overcooked” throughout. In the case of a track that has been recorded too loud throughout and has continuous distortion, not only will the repair take a very long time, but the results are likely to be disappointing because so much of the true audio data has been destroyed and the reconstruction can never recreate it exactly.

9

Hello, gentlemen. Sorry for the delay in posting a status update.

Good news: Steve’s script works very slick. I’ve posted three mp3s here to illustrate the results. The first one is the original 16 seconds I recorded off the web; the second is after processing with the script; the third is a mix, with the clicky original in the left channel and the processed result in the right - headphones make the differences easy to spot.

Not all the clicks were removed. I suppose we can attribute that to absolute-value-AND-ing being a naive (if somewhat useful this time) approach, and to some clicks appearing in both channels of the raw tracks, thus not being filtered out. Regardless, the processed mp3 is significantly cleaner than the original one.

This is a good result for me. While I accept Koz’s criticism re fixing my PC, what I’d really like is to strip it down to bare metal and be selective with the re-installs. I’ll have a new machine sooner, I think. In the meantime, when I have time to prepare, I use my lower-spec laptop to record click-free. But if I’m at my desktop when a target of opportunity streams by, I capture it, preferring to have it clicky than to not have it at all. I managed to put the script into the plug-ins directory, and it’s a terrific addition to my effects menu. Just two (mouse) clicks away when I make a new recording.

I was also listening for a difference in quality between the original and processed tracks beyond simply a reduction in clicks. However, the processed track seemed neither noisier nor more distorted to me than the original. No golden ear here, I guess.

FYI, the first time I ran the script, it took about 85 seconds. After reloading the .aup, I ran the script again, and it took 12 seconds. Perhaps my Firefox session with its 60 open tabs took a while to be paged out the first time.

Thanks again, Steve & Koz. I very much appreciate how quickly and effectively you gentlemen responded to my post.

10

Excellent - it’s worked pretty well.
Looking at the clicks that remain, they are present in both channels, so as expected they have not been removed, but at least there is now a manageable number so they could be removed manually using “Repair” if you wanted to.

Apart from the clicks that are removed, the sound should not be changed at all by the processing - where the sample values are equal in both channels, the sample values are totally unchanged, so the sound quality should be unaffected.
(which is better than using more “conventional” methods).

I think this will be largely down to available memory, but I wouldn’t be surprised by any odd behaviour, it’s just a quick hack really :wink: