Help needed to create a plugin for fixing dv clicks

[janolap1]

Hi,

I’m really new in those forums, but I have used Audacity for years.

I have a problem with a audio track from my dv camcorder (mini-dv sd).
Here is the problem : from time to time, during the video, the sound makes a sort of “click”.

When I import the sound of the video inside audacity, I found that the “clicks” are either left or right.
And I have found that the structure of the click is quite simple :
A sample every 4 or 5 (or 6 or 7) has a db value of zero.

I could fix it by raising the sample to it’s (approximative) original value manually, but it’s an enormous work.

So I thought about creating a plugin for Audacity. I make some programs, but in a C, or php, or basic syntax. But Nyquist has a quite different syntax ; and I would like to work with individual samples (which is not covered by the tutorial).

So I have wrotten a pseudo-code for my plugin. So if I someone could translate it into a Nyquist plugin, it would save me a lot of time (and could help my friends with having an usable video of their show…).

Here is the pseudo code :

for $i=0 to numberOfSample(selection)-1
{
 if (($i>0) and ($i<numberOfSample(selection)-1)) then
 {
  if (getdbSample(selection, $i) ==0)
  {
   putdbSample(selection, $i, (getdbSample(selection, $i)+getdbSample(selection, $i))/2);
  }
 }
}

Where numberOfSample(s) is a function that returns the number of samples in the selection s;
Where getdbSample(s,n) is a function that returns the db value of the sample number n in the selection s;
Where putdbSample(s,n,v) is a function that replaces the db value of the sample number n in the selection s by the value v;

Could you help me with this ?

Cordially

Jean CARTIER

[kozikowski]

This is me watching. As with most problems like this that deal with Real World Problems, you have to determine when to apply the correction. While it’s not impossible for an actual show to go to “all zero bits” for a duration, it’s only unlikely. Similarly, if the samples are not zero, but actual digital damage, then the samples may not conform to the “all zeros” condition for filter application.

I’d get the camcorder fixed myself. How old is it? We have some older SD Mini-DV camcorders in the stable that routinely drop one of the video heads on occasion.

If you have one of the cameras that supports plug-in microphones, you might try plugging and unplugging several times to “clean” the contacts. It’s very unlikely that once side of a digital stereo bitstream suddenly vanishes, it’s much more likely something ahead of the D/A converter is frying.

Koz

[Robert_J_H]

What do you mean by dB value of 0?
0 dB is full power (amplitude 1 or -1) and -inf dB is amplitude 0.
Let’s talk for now in terms of amplitude (-1 to 1). I presume that one sample could be 0.707 the next (drop-out) 0 and then 0.7 again.
This can also be the case for samples under the center line (-0.6 0 -0.8), I guess.
And this happens every 4th to 7th sample? You know, that signal has already lost a huge amount of information. I doubt if the results will be satisfying.
Anyway, here is some sample code with some arbitrary sample values that have such drop outs.

(setf *sr* *sound-srate*)
(setf selected-samples (snd-from-array 0 *sr* (vector
  0.0 .1 0.3 0.5 0.8 0.0 0.7 0.4 0.1 
  -0.3 -0.6 -0.7 0 -0.8 -0.4 -0.2 0.0 0.0)))
(setf correction 
  (diff 
    (snd-chase selected-samples (/ 2 *sr*) (/ 2 *sr*))     selected-samples))  
(setf result (sim (mult 3 Correction) selected-samples))
;; Let's compare the results
(setf before (snd-samples selected-samples ny:all))
(setf after (snd-samples result ny:all))
(setf corr-by (snd-samples (diff result selected-samples) ny:all))
(format t "Before corrected by after~%")
(dotimes (smp(length before))
  (format t "~a    ~a          ~a~%"
    (aref before smp)
    (aref corr-by smp)
    (aref after smp)))

Thats not yet a plug-in, its for demonstration purpose only. Copy the code to the Nyquist prompt, press the debug button and control if the correction somewhat resembles what you have in mind.
If we are on the right way with this solution, there will be no need for loop structures and reading out each sample (that’s very slow and risky in Nyquist).
Snd-chase is a kind of lowpass filter, but not in the usual sense. We use it here to smooth the abrupt changes from high to low and high again within 3 samples.
(If you want to look up any function or command, you can install my plug-in:)
Nyquist/Xlisp Reference

[janolap1]

Thank you for all those replies…

There was an error in my pseudo code :

for $i=0 to numberOfSample(selection)-1
{
 if (($i>0) and ($i<numberOfSample(selection)-1)) then
 {
  if (getdbSample(selection, $i) ==0)
  {
   putdbSample(selection, $i, (getdbSample(selection, $i-1)+getdbSample(selection, $i+1))/2);
  }
 }
}

I’ll try what you said and tell you if I succeed.

[janolap1]

Hi,

To answer Koz, my dv cam is pretty old.
But I think there’s a problem with the 16bit audio recording.
Maybe a problem with the tape. I have to investigate further.

The sample code of Robert : I think, being a newbie with Nyquist langage, that you create a sound from the array and try to correct it.
There is a error when debugging (i’m using audacity 1.3.0-beta) : error: unbound function - DIFF
Ok, using Audacity 2 it runs.

Well, I think this could help, but is there a way to do this :
if sample==0 then sample= (previoussample+nextsample)/2

Let me show you 2 pictures of the samples : one before correction and one after manual correction :

Cordially

Jean CARTIER

[steve]

@janolap1

Well, I think this could help, but is there a way to do this :
if sample==0 then sample= (previoussample+nextsample)/2

Yes that could be done, but Robert is trying to (cleverly) avoid bit-wise processing as bit-wise processing in Audacity is notoriously slow.

Robert’s code will work where there is a sudden large drop in the value of one sample value from the surrounding audio samples, and it will work pretty quickly. Unfortunately I don’t think that it is going to work very well on your damaged audio because the drop to zero is probably not far enough.

If you want to try Robert’s code, this is a short version of the “correcting” part of the code. Select a section of the damaged audio, then copy and paste this code into the Nyquist Prompt effect to run it:

(defun correct (sig &aux (slew-time (/ 2 (snd-srate sig))))
  (sim
    (mult 3 (snd-chase sig slew-time slew-time))
    (mult -2 sig)))

(multichan-expand #'correct s)

[Robert_J_H]

(Thanks Steve)
Hi Jean
The code should also work with your old audacity version since steve has replaced (diff s1 s2) with (sim s1 (mult -1 s2)).
As you can easily test with my example above, the code works also when the drop out isn’t exactly 0 (0.1 works still).
As I said before, the code is not yet on production level, so don’t be surprised when it doesn’t work entirely satisfactory the first time.
Especially the multiplication with the factor 3 is arbitrary and not save as such. Let me illustrate this: If the sample triple is (1.0 0.0 1.0) the calculated correction will be 0.5 x 3 = 1.5 and therefore 4 db to high. It would be wiser to take the square root (one or more times) of the calculated correction. For 0.5 we would get:
0.5 > 0.707 > 0.841 > 0.917 …
You see, we will come pretty close to 1 the higher the fractal power is. Unfortunately, this won’t work for negative values. We therefore have to separate the lower and upper parts of the wave form and recombine them in the end.
But first, let us know if the code improves the sound at least partially.

[steve]

If you only need to process very short sections at a time (say, under a couple of seconds), then your algorithm can be applied quite simply.
I’ve added a lot of comments to this code so that hopefully you can see what it is doing.
Note that the sample values must be exactly zero to be corrected.

; limit selection to 100,000 samples
(setq max-samples (min 100000 (truncate len)))

;;; Function to correct zero samples
(defun correct (sig)
  ;grab samples in an array
  (setf s-array (snd-samples sig max-samples))
  ;; main loop
  (do ((count 1 (1+ count)))
      ; loop through samples
      ((>= count (- max-samples 2))
        ; return sound from samples
        (snd-from-array 0 *sound-srate* s-array))
    ; when sample value is zero
    (when (= (aref s-array count) 0)
      ;; set new 'average' value
      (setf (aref s-array count)
        (/ (+ (aref s-array (1- count))
           (aref s-array (1+ count)))
          2.0))
      ; and skip the next 3 samples
      (setq count (+ 3 count)))))

;; check selection length and if OK run the program
(if (<= len max-samples)
  ; run it
  (multichan-expand #'correct s)
  ; print an error message
  (format nil "Error.~%maximum selection length is ~a seconds."
    (/ max-samples *sound-srate*)))

Again you can run this code in the Nyquist Prompt effect.
Don’t try to use this simple approach on selections more than a few seconds or you will probably crash Audacity.
This version is limited to 100,000 samples which should be safe.

[steve]

Just a little note that I didn’t comment:
The plug-in assumes that is a zero sample is found, the next three samples are not faulty, so it skips them by adding 3 to the counter. That’s just to make it run a little quicker since you said that the zero samples only occur every 4 or more samples.

[janolap1]

Hi and thank you !

You’re great.

Using robert’s code, the effect expected is not reached.

I have adapted Steve code like this to make it work on my sample :

    ; limit selection to 100,000 samples
    (setq max-samples (min 100000 (truncate len)))

    ;;; Function to correct zero samples
    (defun correct (sig)
      ;grab samples in an array
      (setf s-array (snd-samples sig max-samples))
      ;; main loop
      (do ((count 1 (1+ count)))
          ; loop through samples
          ((>= count (- max-samples 2))
            ; return sound from samples
            (snd-from-array 0 *sound-srate* s-array))
        ; when sample value is zero
        (when (< (abs (aref s-array count)) 0.01)
          ;; set new 'average' value
          (setf (aref s-array count)
            (/ (+ (aref s-array (1- count))
               (aref s-array (1+ count)))
              2.0))
          ; and skip the next 3 samples
          (setq count (+ 1 count)))))

    ;; check selection length and if OK run the program
    (if (<= len max-samples)
      ; run it
      (multichan-expand #'correct s)
      ; print an error message
      (format nil "Error.~%maximum selection length is ~a seconds."
        (/ max-samples *sound-srate*)))

And it works !

So now, I have to isolate every click and apply this nyquist patch !

Thanx a lot !

I have to learn Nyquist now…

[steve]

@janolap1

Using robert’s code, the effect expected is not reached.

But it was a very neat idea.
As a rule it is better to give the job of looping through samples to the C++ code (as in Robert’s script) rather than looping through samples in LISP (as in my script).

An alternative method could have been an OOP approach of creating a sound object that remembers the last sample value, then if a sample value of zero is found, the previous value is substituted. This would probably not produce as smooth a result as the current code, but it would probably be faster and not suffer from the limits to the duration that can be processed.

A couple of points about your modification:

1. You should really change the comment on line 21 to “; and skip the next sample”.
   If you don’t wish to skip any samples, you can miss out lines 21 and 22 altogether (just ensure that the parentheses match).
   Not skipping samples opens up an option to improve the accuracy of the effect (see below).

2. (line 15) Did you determine the value 0.01 (-40 dB) by experimentation?

A quick way to check sample values - run this in the Nyquist Prompt and press the Debug button to see the output. This will print out the sample values in dB for up to 100 samples (mono tracks only):

(dotimes (i (truncate (min len 100)))
  (print (round (linear-to-db (abs (snd-fetch s))))))
"Results in Debug window."

(for a more advanced analysis tool, see: http://manual.audacityteam.org/o/man/sample_data_export.html)

You could then use at line 14 - 15, for example,

; when sample value is below -40 dB
(when (< (abs (aref s-array count)) (db-to-linear -40))

To make the correction more accurate, you could avoid “correcting” samples where the waveform crosses the zero line.
This version checks every sample value, but ignores low amplitude samples that are during absolute silence or at zero crossing points.
The debug window prints the number of samples that have been corrected:

; limit selection to 100,000 samples
(setq max-samples (min 100000 (truncate len)))

;;; Function to correct zero samples
(defun correct (sig)
  ;grab samples in an array
  (setf s-array (snd-samples sig max-samples))
  ; number of samples corrected
  (setq total 0)
  ;; main loop
  (setf output
    (do ((count 1 (1+ count)))
        ; loop through samples
        ((>= count (- max-samples 2))
          ; return sound from samples
          (snd-from-array 0 *sound-srate* s-array))
      ; when sample value is below -40 dB
      (when (and (< (abs (aref s-array count)) (db-to-linear -40))
              (> (* (aref s-array (1+ count))
                   (aref s-array (1- count)))
                 0))
        ; increment total
        (setq total (1+ total))
        ;; set new 'average' value
        (setf (aref s-array count)
          (/ (+ (aref s-array (1- count))
             (aref s-array (1+ count)))
            2.0)))))
  (format t "Number of samples corrected: ~a~%" total)
  output)

;; check selection length and if OK run the program
(if (<= len max-samples)
  ; run it
  (multichan-expand #'correct s)
  ; print an error message
  (format nil "Error.~%maximum selection length is ~a seconds."
    (/ max-samples *sound-srate*)))

@janolap1

So now, I have to isolate every click and apply this nyquist patch !

You may find that you can see them more easily using the Spectrogram view http://manual.audacityteam.org/o/man/track_drop_down_menu.html#Spectrogram