Killing hard clipping

Special Interest Groups Audio Processing
1

I’ll tell you now, I don’t know anything about audio programming.
Clipfix hardly does anything for hard clipping.

What is the solution to hard clipping?
I think I’ve figured out what needs to be done.

In a hard clip, the low frequencies are distorted and limited, creating harmonics that go into the high frequencies. The high frequencies that existed already are clipped out of existence because of the flat tops.

I think I know how this would work:
First, the high frequencies created by the clips are removed by replacing the wave tops with a natural formation (created by linear prediction?). They are then replaced with the good frequencies present outside of the clipped areas, like a bridge, which I think could be interpolated.

Tell me, do I have any idea what I am doing?

2

Clipfix works by cubic spline interpolation over the truncated portion of the waveform. Unfortunately that’s about as good as it gets. The correct “solution” is to avoid clipping in the first place.

3

I don’t mean to sound rude, but doesn’t the usually work only on sine waves and stuff?

For the most part, it seems to clip the tops of the waves even more, seriously. Only occasionally does it actually put a suitable top on the clip; when it does actually put one there it usually overshoots, too.

There has to be a better way.

EDIT: If there isn’t a technically correct way, there has to be a way to make it at least sound like there is less clipping

4

Clipping is one of the things you can do to kill your show.

There is still a short list of things we can’t fix. That’s one of them. Clip Fix works by best guess approximation. It’s not unusual for it to reconstruct the waveform and have the show sound worse.

Fix the clipping at the capture stage.

Don’t have big-room echos in your show. We can’t fix that, either.

Koz

5

I don’t have a show or anything that I am angry about, I’m just thinking about this in general.

So just to clarify, the clip fix attempts to remove clipping by interpolating the clipped top of the waveform.

I think I’ve found a rational way of saying what I want to.

Instead of fixing just the high frequency tips of the clipped areas, I think it could also try to fix the bass signal that ends up clipped as well.

I don’t really see how it is possible, but I think if there was some way to identify the harmonics associated with a clipped sine wave something could be done.

EDIT: There is a visible spike in the high frequencies whenever a clip occurs

I understand your viewpoint now that I’ve done some research.


EDIT 2:

So, my proposal is that if the places where the clipped tops of lower, louder waveforms could be identified, the according top of a sine wave could be added to the area of that clipping.

6

I’m pleased about that - I hate it when we have to try and break it to people that their damage recording is irreparable, especially when it is for them a really important recording.

Sound engineers go to great lengths to avoid certain problems with their original recorded material. One is to avoid clipping and the other is to avoid excessive echo / reverberation. The reason that they take such pains to avoid these two particular problems is because they are notoriously difficult to repair satisfactorily.

If there’s just a small amount of clipping from the tops of a few peaks, then this can be repaired reasonably well, and ClipFix does a reasonably good job with that. If the audio is severely distorted, then it’s time to start again. I’ve had the opportunity to use some very expensive sound restoration equipment, and on very slightly clipped audio the equipment performed brilliantly, but with badly distorted audio the results were worse than ClipFix, (and sounded worse than the original “un-fixed” audio).

I’ll try and give an idea of the problem. In these two tests, the numbers represent sample values, but the values have been “clipped” at 9.

Correct the sequence test 1.
1, 2, 0, 4, 7, 9, 9 , 9, 8, 7, 6, 7, 8,

Correct the sequence test 2.
0, 5, 9, 9, 9, 9, 4, 5, 4,

In test 1, there is a pretty good chance that the middle “9” should actually be a “10”.
In test 2, we really have no idea what any of the 9’s should be. Do we guess at 9, 12, 11, 9? Can we be sure that any of the 9s are correct?

ClipFix works by looking at the “slope” either side of the clipped region, so in the case of test 1, it can calculate that the middle “9” is probably lower than it should be, and will replace that value with a higher value. In the case of test 2, there are big increases between the first three numbers, so ClipFix will assume that the next digit should also be substantially higher, but in reality there is no way of knowing what it should have been. To complicate the matter further, if the “clipping” has occurred in the analogue domain, the peak values are probably highly compressed before the clip limit is reached, so even if the clipped region is perfectly restored, there will still be substantial distortion due to the values below the clip threshold being wrong. Complicating the matter even further, clipping is most likely to occur during suddenly high “transients”, for example a drum hit. In this case, the harmonic content of the clipped region will have virtually no relation to the harmonic content of the sound immediately before the drum hit.

Here are a couple of spectrograms of a low pitch drum being struck. The upper track is a clean recording and the lower track is slightly clipped. We can see that on the second tracks there are a series of broadband pulses after the initial strike. How is an automatic repair algorithm to know if those frequencies should be present or not?
trackpanel000.png

7

Thank you for the breathing room.

You make a good point when you say that the problem of additional frequencies is very difficult to fix. There are other things that occur during clipping, though.

I do have another thing that could be done, though.
We are aware that people do not hear sound waves so much as they hear frequencies, so this is a fairly simple example:

This is a noise signal which is clipped, notice the gap in the high frequencies?
I think that this is a small enough length of audio that an inaccurate fix that simply reduces the jarring results of this would be much better.

Also, the Declipper does not assume that the entire clipped area is above the threshold. It could be much more accurate using a system that only allows for the waveform to be above or equal to or at the clipping threshold.

What I’m basically saying is, clipping is caused by a waveform but it has harmonic consequences which can be reduced.
The spectrum analyzer does not know that the waveform is clipped
The waveform analyzer does not know that the spectrum has a gap in it
If we use an ideal combination of both, it will work the best.

Though we cannot remove the common cold, we can use medicine to reduce the symptoms.

8

clipping is caused by a waveform but it has harmonic consequences which can be reduced.

Which assumes that the original musical note had no harmonics of its own. A trumpet comes to mind. By actual experience it’s terrifically hard to mic a trumpet without some sort of distortion and/or clipping. Unmuted trumpet notes have their own distinctive harmonic patterns and clipping would simply sub bad ones for good ones. On some damaged performances it’s rough to tell what actually happened until you drag the scope out.

“Oh, right. That’s where Donny really leaned into the b flat, wasn’t it?”

Koz

9

Koz, I understand what you are saying.
When I say harmonic consequences, I’m not saying that we’re working with pure tones that we’re trying to restore, I’m saying that clipping produces additional harmonics. I’m not suggesting that we somehow sift through the spectrum and figure out how to clean out the clipping, though if it is was possible I would do it. I don’t take insult; thanks for helping me clarify.

Also, I do not advocate clipping and then just using whatever clip fixer no problems


Anyway, here is what I was writing, but summarized:

So, first the clipped top would be replaced with a natural curve that reduces the sound of the clipping, then the spectrum would be interpolated to put a better sound back in.

The first thing would use interpolation, but it would use a method that only allows the waveform to go outside the clipped threshold.
I would then use a spectrum analyzer to stretch, using bi-linear interpolation, the edges of the spectrum around the clipped area

How does that sound?

I am also thinking that if somehow the two channels could reference each other, if they could work together, though I don’t see exactly how it would work, it’s just another thing to think about.

EDIT: After doing some tests, clip fix performs reasonably well with reducing added harmonics when set to 99%
I think it should be used for the interpolation, just also with the whole threshold whatnot

10

There is a (non-free) program called [u]SeeDeDlip[/u]. And, I believe Adobe Audition has a clip repair tool.

I have no idea if these tools are more sophisticated than Clip Fix. There might be a way to extract some higher-frequency information from the non-clipped part of the waveform. The images on the SeeDeClip website, appear to show something like that.

11

Nice one DVDdoug. I’ve just been testing that and it’s very impressive on the supplied examples. It didn’t cope with heavily distorted analogue clipping, but then nothing does.

12

I did some testing on 30 seconds of audio, which was the limit.

The first thing was that it took like and instant, which is unnerving. It makes you think they are using some other trick that just makes it look better or something.

The second thing was when I looked at it. It looked as though the clip fixed areas were pushing up against some other threshold. The audio also generally had an appearance of being crushed inward, like reverse clipping (where the waveform has the inner part cut out of it)

When I listened to it, it was clear.
Very little noticable difference, it sounded scratchy, like the bitrate had been lowered or something, and though the waveform looked better, all of the clipping was obviously not compensated for.

I give it 0/1
I though it might do something, but it didn’t.


Speaking of all this, I have thought of something interesting regarding a real solution.
When a higher waveform is mixed with a low one, the high waveform tends to clip on the top or bottom until it is pushed out of existence, while the lower one generally also clips. However, the top of said waveform will be mislead a waveform declipper because the low frequency “deforms” the high one. It needs to follow the high and low waveforms separately.

You see the apparent issue here.

Geez, fixing this is a real stack of problems, isn’t it?

13

Yes there is - Avoid clipping in the first place.
A question to mull over - how would you correct a clipped square wave?

14

I’m saying that clipping produces additional harmonics.

It doesn’t always. If you reference the original rich trumpet blast and clip it just the right amount, the rich but powerful musical overtones vanish and are replaced by straight old harsh square wave every third to infinity harmonics – not necessarily higher than the original. It may sound like that. Ears don’t like third harmonics.


There is a developer who makes a really nice volume compressor/manager as a plugin for Audacity. Chris’s Compressor. It’s pretty amazing how well it works. He designed it to be musical – to change the volume expression in opera so he could listen in the noisy car. So instead of the left-brain math coming first, he started with a right-brain creative task and worked “backwards.”

The only hope of making anything like this work is create software that “knows” melodic progression and harmonious music. Anybody who starts the process – step one – with “…the delta rate of change of the remaining Fourier sampled slope…” is already walking dead.

I bet you could go a very long way to appearing to solve the problem by rapidly switching a low pass filter in during the clipped passages. I can see the horrified looks all the way from this time zone, but I bet that would sound better than the math exercises.

Koz

15

Koz, I feel like I’m not being clear enough.
I have stated a lot of scatterbrained thoughts, and I think it’s about time I go it alone.

Can anybody suggest a way I could learn how to make plugins compatible with Audacity?

I’m beginning to feel like this would be a better solo project, because I don’t want to slave a programmer into making something based on my wild theories, and I’d rather be right or wrong alone rather than fighting a battle that is very hard to prove either side of.

If I get anything, I will share it with everyone.

16

Probably the easiest type of plug-ins to make are “Nyquist plug-ins” http://wiki.audacityteam.org/wiki/Nyquist_Audio_Programming
(ClipFix is a Nyquist plug-in)

17

I’m looking in to it, it looks very promising to use Nyquist for this.

The tutorials have helped me quite a lot, too.

18

Well, I like, if not love it.
It’s just the the manual assumes that I already know LISP or something.

Any way I can get started with learning this?

19

See here: http://www.audacity-forum.de/download/edgar/nyquist/nyquist-doc/xlisp/xlisp-index.htm

You will also find a lot of code examples in this part of the forum: http://forum.audacityteam.org/viewforum.php?f=39
If you have any questions about Nyquist code you can post in that section.

20

I have really been meaning to learn Nyquist…

I am looking over my previous posts, and I was totally unclear. I didn’t know what I was talking about, at all.
Here is my method explained using terms that make sense.

The wave shape of the audio will be analyzed and then compared to what it would be if it was straightly-hard clipped (We will figure that part out later), and then corrected to match that.

Each clipped area will be identified as a clip; the clip starts at the first clipped sample, goes through all samples; clipped or not, and ends at the last clipped sample before the wave goes downward past the middle.

The width of a clip at any point will suggest a frequency. Everything above that frequency will be replaced, but only within the clipped area. A Fourier transform will be applied and the new harmonics will be determined by interpolating by them from the non-clipped areas.

Then, at a user specified level, a dynamic algorithm will be applied to the entire mix revive the general sound of the audio before the clipping. The percentage of samples that are clipped in each area will be used to determine how loud to the make the area. This will help the punch of the sound and may reduce clipping-related limiting, but will not be needed in all situations.

How does that hold up?