Improving "Clip Fix"

[steve]

ClipFix is a Nyquist plug-in.
While a very versatile language for rapid development of plug-ins, Nyquist is very slow for sample-wise processing and would probably be too slow to implement a high order least squares auto-regression.

I would therefore like to suggest that the interpolation used in "Effects"->"Clip Fix" be replaced with the interpolation used in "Effects"->"Repair."

IMO that would be possible for a built-in effect, and could provide very good results provided that it was used where each damaged area was small enough and surrounded by undamaged audio. As shown in my previous post the Repair algorithm fails dismally if these conditions are not met.

Accurate detection of clipped audio is extremely difficult except in the special case of "hard" (digital) clipping of uncompressed audio data.

I think it could be difficult to find a developer that is willing to spend the required time and effort on an effect that will only work in cases where all of these prerequisites are met.

[Quant]

IMO that would be possible for a built-in effect...

That's actually what I was hoping for.

...and could provide very good results provided that it was used where each damaged area was small enough and surrounded by undamaged audio. As shown in my previous post the Repair algorithm fails dismally if these conditions are not met.

As far as I can see, Repair only fails dismally when a damaged area is surrounded by other damaged areas that occur periodically with a frequency similar to the fundamental frequency of the underlying signal. In all of the cases where I have used Repair on an interesting signal, like music ripped from a CD or my own recordings, this rarely happens. When it does happen, though, this would be easy to check for. If the result of a Repair operation leaves the processed area largely unchanged (or the area is larger than 128 samples), fall back to cubic interpolation. Also, we can make multiple passes over the entire track for better quality. On the first pass, we could use Repair (with cubic interpolation as a fallback, like I just mentioned), and on subsequent passes, we can refine the previously processed areas with Repair.

[Gale Andrews]

In this final example, we have a region selected that included "not clipped" waveform, surrounded by clipped waveform - After Repair shows that the "not clipped" waveform has been clipped (so as to match the surrounding waveform). This illustrates why we can't just iterate across a longer damaged region 128 samples at a time.

My issue with recommending Repair as a clipping repair tool for significant selections of the waveform is exactly that. Of course, patience wears thin, and users will resort to repairing blocks of 128 samples that "mostly" contain damaged audio (or they will fail to identify the samples that are not of the expected pattern). In those cases Repair gives completely wrong results. Clip Fix rarely gives completely wrong results (I would not call the result in Quant's example "bad").

Do you have examples of music where Clip Fix is worse? I'm not doubting they exist.

I would therefore like to suggest that the interpolation used in "Effects"->"Clip Fix" be replaced with the interpolation used in "Effects"->"Repair."

IMO that would be possible for a built-in effect, and could provide very good results provided that it was used where each damaged area was small enough and surrounded by undamaged audio

To me, asking for Clip Fix to use the Repair algorithm is still tantamount to asking for Repair to work automatically on longer selections (otherwise, why not carry on painstakingly using Repair as now)?

If so, Repair has got to detect the clipped regions inside the selection for itself (correctly). If it does not do so correctly the result will be worse than Clip Fix.

You seem now to suggest Repair and Clip Fix could be combined on a "try the other algorithm if necessary" basis, but I wonder how fast the processing will be?

Also the two effects are somewhat different tools in my view. Clip Fix can also be used almost like a waveshaper to construct peaks in audio that isn't clipped but perhaps overcompressed. I quite often use it like that - illegitimate or not I think it does a great job in those cases to breath life back into the audio (and much easier to use than a conventional expander).


Gale

[steve]

Clip Fix can also be used almost like a waveshaper to construct peaks in audio that isn't clipped but perhaps overcompressed. I quite often use it like that - illegitimate or not I think it does a great job in those cases to breath life back into the audio (and much easier to use than a conventional expander).

Interesting. I've never tried that. I'd guess that it would be most "natural" on audio that had been heavily "limited" - something which is very difficult to combat by other means. I'll have to try it

To me, asking for Clip Fix to use the Repair algorithm is still tantamount to asking for Repair to work automatically on longer selections (otherwise, why not carry on painstakingly using Repair as now)?

I think there is a distinction between the proposals.

I don't recall seeing this in Audacity, but according to the code:

Code: Select all

This was formerly the PopClickRemoval effect, but it was
renamed and focused on the smaller subproblem of repairing
the audio, rather than actually finding the clicks.

So originally it looks like it was intended to be a two pass effect - first detect damaged audio, then fix it. That approach is like ClipFix, but ClipFix uses a repair algorithm that does not require as much "good" audio between the "bad" parts.

Asking Repair to work automatically on longer sections than now is easy to program. The problem is that it becomes increasingly slow. With only a modest increase in size it could take several minutes to process each selection. With the amount of increase that most users want you could be waiting for hours.

As far as I can see, Repair only fails dismally when a damaged area is surrounded by other damaged areas that occur periodically with a frequency similar to the fundamental frequency of the underlying signal. In all of the cases where I have used Repair on an interesting signal, like music ripped from a CD or my own recordings, this rarely happens.

I doubt that is true, though it may appear to be the case.
Looking casually at your "Clipping Example" there appears to be one bad section every 210 samples or so, but that is because you are intelligently grouping together multiple regions of bad samples that have a few good samples mixed in. It is very easy to "see" the groupings, but much harder and slower to write an algorithm that can make the grouping sufficiently intelligently to fit the general case (rather than just one or two specific cases).

clipped.png (10.76 KiB) Viewed 2766 times

[Quant]

I doubt that is true, though it may appear to be the case.

I'd be very interested to see a counter-example. And, in case it isn't clear, I'm referring to each individual damaged section, not groupings of sections.

...but that is because you are intelligently grouping together multiple regions of bad samples that have a few good samples mixed in.

Most of the clipping that I've encountered in real life appears just as you described. I wasn't trying to be intelligent. I was simply trying to create an example that could be viewed as a simplification to real-world audio, not just a waveform that is (or was, prior to clipping) essentially only one frequency (...when do you ever see that in real life?).

It is very easy to "see" the groupings, but much harder and slower to write an algorithm that can make the grouping sufficiently intelligently to fit the general case (rather than just one or two specific cases).

I'm not exactly sure what you're trying to show here, but I think that the algorithm described in my previous post would work quite well.


I'd also like to mention that there have been many times when I have fixed clipping using Repair and the results sounded as if there had never been any clipping in the first place. I have never had this happen using Clip Fix. I'm not trying to insult Clip Fix; I just really wish there were an easier way to automate Repair on multiple sections. I'm not even asking for any advanced clipping detection. What Clip Fix uses for detection is already good enough for my purposes. I understand that the developers are busy and that they're doing this on their own time for free. If I haven't made my case by now, I guess I'm just alone on this (and that's okay ). I'll probably end up writing the algorithm myself (which I would be happy to contribute, if someone would like to port it to work with Audacity, assuming there were any interest).

[steve]

OK, I'm with you now Provided that your method is followed rigorously it works pretty well and certainly better than ClipFix.

I'll probably end up writing the algorithm myself

Do you/can you program in C++ ?

[Edgar]

Sorry to come so late to this…I'm "away from my desk" until at least 27 June and if I get really really lucky, the end of August! I too find Repair produces more pleasing sounding results and have already made some modifications to the source code (for my personal use) that might have some bearing on this topic. I thought I had posted the code here but I can't find it; it does a couple of things: 1) adds a new preference item giving the user control over the maximum number of samples Repair will accept, 2) if the user selects committee samples a dialog opens specifying the maximum, the number selected & the difference (selected minus maximum).

When I'm available again I would be very happy to help out with the coding (just don't ask me to write a new algorithm).

[steve]

@Edgar
1) Do you know anything about writing LADSPA plug-ins (and making them work on Windows)?
2) Have you tested "Repair" for the effect of changing the maximum order of the least squares regression (currently hard coded at 50)? I would expect a significant increase in speed if that number is reduced and a slight improvement in quality when increased. If that is correct then it could be a very useful parameter for what is being proposed.
3) I can probably help with a detection algorithm.

[Quant]

Do you/can you program in C++ ?

Yes, that is my language of choice. I've designed and implemented quite a few signal processing algorithms for personal use, most having a simple command-line interface. Unfortunately, I probably won't be able to get too familiar with Audacity's internals, but I'm pretty sure that getting any code I write to work with Audacity would be very minimal. Also, I would use existing Audacity functions, like InterpolateAudio.

[Edgar]

@Edgar
1) Do you know anything about writing LADSPA plug-ins (and making them work on Windows)?
2) Have you tested "Repair" for the effect of changing the maximum order of the least squares regression (currently hard coded at 50)? I would expect a significant increase in speed if that number is reduced and a slight improvement in quality when increased. If that is correct then it could be a very useful parameter for what is being proposed.
3) I can probably help with a detection algorithm.

No LADSPA experience, sorry. Have not looked at that code. I'm on the road with an iPad so have no good access to the code now.