Steve's Updated ESP-clipfix

Hi Steve,

Recently came across your CLIPFIX MOD Nyquist script and was excited to try it out.
However, it seems to do nothing and gives a warning about “cannot go back in time” or crashes audacity.
I did try it on a mono track.

Running Audacity 2.3.1 on MacOS.

Any ideas?

Thanks.
Paul.

Please post a link to where the plug-in is.

Here you go Steve:

http://goo.gl/tLKdv6

OK, so that’s this one from 2012: https://forum.audacityteam.org/t/clipfix-see-waveform-pic-plugins-available/4171/72
I’ll take a look.

Thank you very much Steve.

EDIT:

Just read the thread you linked to and the user seemed to have good results by selecting
2 second sections.
Will try that too, currently my selections are about 3-4 minutes in length.

This is where it started, back in 2008 :astonished:
https://forum.audacityteam.org/t/clipfix-see-waveform-pic-plugins-available/4171/10

Only 13 years…a mere blink of the eye :smiley:

Had a giggle at your post:

Other than playing with the Nyquist prompt in Audacity I’ve not done any Lisp/Nyquist programming…
… It’s about time that I started doing a bit more with Nyquist

I would say that in the 13 years, you have done quite a bit of “playing” with Nyquist. :smiley:

It seems that it never went past being a “proof of concept”.

Meanwhile, there were many changes to the shipped ClipFix effect, including a total rewrite to fix multiple problems (https://github.com/audacity/audacity/commit/a9879bddf0fbfad38784d276d5a80a1e5fd45cfc#diff-ce788ce90d9e6ff88800322693e1f06278f7a499ec21d0198255780af4ffe359)

Basically what the “ESP-ClipFix.ny” does differently from the shipped version, is that it does two passes.
The first pass looks for inverted peaks, and replaces the inverted audio with a flat top to the peak, like this:

Tracks000.png
This is the code that does that (updated for current versions of Audacity).
Note that this has a length limit of 1 million samples.

;specify max length to avoid crashing
(setq slength (truncate (min len 1000000)))

(setq limit 0.5)  ;maximum diference between consecutive samples.

(defun fix-inversions (sig)
  (let ((array (snd-fetch-array sig slength slength))
        current
        next)
    (setq current (aref array 0))
    (dotimes (i slength)
      (setf next (aref array i))
      (cond
        ((> (abs (- current next)) limit)
          (setf (aref array i) current)
          (setf next current))
        (t (setf current next))))
    (snd-from-array 0 *sound-srate* array)))

(multichan-expand #'fix-inversions *track*)

Wow that was quick, thank you.

…looks for inverted peaks, and replaces the inverted audio with a flat top to the peak…

Exactly what I was looking for.
What I will probably do is use the code you just posted and turn that into a plug-in.
Will call it “Invert-Peaks.ny”.

This way, if I only need clipfix can use the standard one, but if the audio has inverted peaks,
then will run your plug-in first then clipfix.

That way, no need for extra computing time if not required.

Noted about the 1 million sample limit.

Thanks again.

It works pretty well…
Screen Shot 2021-10-05 at 10.23.41 PM.png
Just wondering, any advantage to adding a slider to allow the fine tuning of
“limit” i.e. maximum difference between consecutive samples.
Perhaps a range of 0.1 - 0.9 ?

Change the line:

(setq limit 0.5)

To:

;control limit "Sample Max Delta" real "" 0.5 0.1 0.9

EDIT:

Just tried it and values other than 0.5, does give some strange results at times, but going to leave it in
as it may just come in handy depending on duration and levels of the inverted peaks.
It defaults to 0.5

EDIT 2:

Going to do more testing and will attach the completed “stand alone” plug-in here for those that may want to use it.

Yes, I think that would be useful (and is probably why I defined it as a variable rather than just a hard coded number).

or perhaps “Fix-Inverted-Peaks.ny” to make the purpose clearer.


This version can handle much longer selections, up to about 100 million samples.
You may want to use the “maxlen” header (https://wiki.audacityteam.org/wiki/Nyquist_Plug-in_Headers#maxlen)
or throw an error message if the selection is too long (The number of selected samples is the value of the global variable LEN).

(setq limit 0.5)  ;maximum diference between consecutive samples.

(setf blocksize 100000) ;size of array blocks
(setf blocklen (/ blocksize *sound-srate*)) ;length of block (seconds)
(setf last-block (rem (truncate len) blocksize))


(defun fix-inversions (ar size)
  (let ((current (aref ar 0))
        next)
    (dotimes (i size (snd-from-array 0 *sound-srate* ar))
      (setf next (aref ar i))
      (cond
        ((> (abs (- current next)) limit)
          (setf (aref ar i) current)
          (setf next current))
        (t (setf current next))))))

(defmacro sim-at (sig1 sig2 t1)
  `(setf ,sig1 (sim ,sig1 (at-abs ,t1 (cue ,sig2)))))

(defun next-array (sig step)
  (snd-fetch-array sig step step))

(defun process (sig)
  (let ((out (s-rest 0))
        (count (truncate (/ len blocksize)))
        repaired)
    (dotimes (i count)
      (setf repaired (fix-inversions (next-array sig blocksize) blocksize))
      (sim-at out repaired (* i blocklen)))
    (cond
      ((> last-block 0)
          (setf repaired (fix-inversions (next-array sig last-block) last-block))
          (sim-at out repaired (* count blocklen)))
      (t out))))

(multichan-expand #'process *track*)

OK, all good points.
Will add them and post back.

Attached, version 1 of the modified “fix-inverted-peaks” nyquist plugin.

I have made a few modifications as follows:

  • Max sample length is set to 100 million.
    (at a project rate of 44.1 KHz, this equates to just over 37 minutes of audio using 1 channel).

  • The maximum sample difference (delta) can be varied from 0.1 - 0.9

  • Added a 5 Hz highpass filter as sometimes the resulting clip can contain some DC offset, due to the flattening of the waveform.
    The 5 Hz HPF should not negatively impact on any music or speech as there is very little (if any) energy down there.
    It does however, block any DC component.

Normal use:

Normally, this plugin would be used on audio that has inverted peaks due to clipping/crushing.
The single control slider (“Sample Max Delta”) can be left at the default of 0.5 for most applications.
For extreme cases, it may be worthwhile experimenting with different settings.

Other Uses:

It can also be applied to isolated drum tracks (and even electric guitars).
In this case, experiment with the “Sample Max Delta” slider for best effect.
Depending on it’s setting, it adds a bit of distortion which can make drums sound punchier (and more crunchy)
and when layered with the original drums, adds some body.

Same goes for electric guitars, a setting of around 0.1 - 0.3, tends to make the guitar sound a bit “brighter”.

Steve, when you have some time, if you could go over it and give me your feedback.
Will then make any changes that you may recommend.

Thanks.

Version 2 below which now includes anti-aliasing filter.
Fix-Inverted-Peaks.ny (1.51 KB)

I discovered that under certain circumstances, applying the plugin, created some frequencies very close to the Nyquist frequency
(half of the sample rate), which can cause aliasing.

This can be very clearly seen below:
Screen Shot 2021-10-06 at 6.41.40 PM.png
To reduce the possibility of aliasing, version 2 of the plugin, includes a 6 pole low pass filter set at sample freq divided by 2.1.
So at 44.1 KHz, the plugin will filter the top end from 21 KHz and at 48 KHz, it will be just under 23 KHz.

The result, is a much cleaner top end:
Screen Shot 2021-10-06 at 6.57.14 PM.png
The original download link for the plugin has been updated with version 2.

As a standalone effect, there’s no problem adding the filters to the code, but if used as originally intended (as pre-processing before applying ClipFix), the filters change the shape of the peaks, causing ClipFix to be less effective.

In this screenshot, both tracks started with the same audio with inverted peaks (the unprocessed sound shown on the left).
The right side of upper track was processed with your new code.
The right side of the lower track was processed with the code that I posted.

Observe that the tops of peaks processed by your code are sloping:

Tracks001.png
And this is after applying ClipFix with default settings:

Tracks002.png

Perhaps make the filters optional ?

Hi Steve,

Thanks for your feedback.

Indeed, the sloping is certainly a by-product of filtering.
Making the filters optional is certainly an option, however, the question has to be asked though…
Which do you believe is the lesser of the two evils? No filtering and a chance of DC and aliasing
or the option of switching them off and have slightly better results with clipfix?
What I’m worried about is, the average user will have no idea about the ill effects of DC and aliasing and simply leave
the filters always off.

If you are trying to repair clipped audio that has inverted peaks, then without doubt it is much better to not filter, so that ClipFix works properly.

If you are using the new plug-in as a standalone effect, then having the filtering enabled is probably better.
If this is your intended audience / use, then you could enable the filters by default.

fip.png
Something like this:

...
;control limit "Sample Max Delta" real " " 0.5 0.1 0.9
;control dcf "DC Filter" choice "Enabled,Off" 0
;control antialias "Antialias filter" choice "Enabled,Off" 0
...

(cond ((and (= dcf 1) (= antialias 1))
        (lowpass6 (highpass2 (multichan-expand #'process *track*) 5) cutoffhi))
      ((= dcf 1) (highpass2 (multichan-expand #'process *track*) 5))
      ((= antialias 1) (lowpass6 (multichan-expand #'process *track*) cutoffhi))
      (t (multichan-expand #'process *track*)))

OK, I have an idea, will post it shortly.

“Traditional” Clip-Fix would try to repair one clip point and do it by comparing the healthy work before and after. It would repair to the logical projection of the waveform shape whether or not it had anything to do with the show.

What everybody wanted was to repair the two hour long clipped and overloaded Senator’s Interview

Are we there yet?

Are we there yet?

Mom!!!

There is a hardware solution to some of these problems (beside making all the microphones quiet). There is at least one sound recorder I know of which makes two recordings, each at different volumes.

Koz