"Wave Stats" plug-in

[flynwill]

@steve

Unfortunately if you go much bigger than that, Audacity will crash.
The problem is snd-maxsamp which computes the samples in memory and retains them until the script completes. As Nyquist is purely 32-bit, I think that gives you a limit of 2 GB before it explodes. This problem is not unique to your plugin, it affect many (most?) Nyquist plugins and there is no way around it until we improve the way that Nyquist accesses the audio.

Probably better to use PEAK and set a reasonable limit.

Yes I figured there might be a limitation there somewhere.

This code uses iterated snd-avg calls to produce the average and get around the max block-size limitation that the previous version had. I did try writing the brute force loop through all the sound sample and accumulate s, s^2, smax and smin but that was going to be a LOT slower than the iterated snd-avg functions.

I also worry a bit, there may be an accuracy problem still. If you naively add up 2^20 samples in 32-bit floating point hoping to generate an average, toward the end of the sum you may be discarding a lot of bits from your samples. I don’t know how snd-avg is implemented – do you know if it has anything that might mitigate this source of error?

Still what would be really nice would be a primitive call written in C++ (or whatever the native language of Audacity is) that efficiently went through a sound and return the sum of the samples, the sum of the samples squared, the max and the min.

[quote=“flynwill”]
However the progress bar is totally confused by the multiple passes that the code takes.
[/quote]

I think what is confusing the progress bar is the time taken to load and release memory. The actual processing is very fast, and that is shown reasonably accurately by the progress bar, but writing hundreds of MB of data to ram, (and then releasing the ram), takes time that the progress indicator cannot take account of.

I found a brief discussion of next cut at plugins being able to provide feedback to the progress bar to make it better… did that ever go anywhere?

There is another problem which is that A-weighting filter is specific to 44.1 kHz sample rate. If, for example, you analyze white noise with a peak of 1.0, the A-weighted result shows -7.1 dB, which I think is about right, but for a sample rate of 192 kHz it shows -12.85 dB. Fortunately the Nyquist filters are clever enough to recalculate their parameters so that the “shape” of the filter remains correct, so it’s only the “fudge factor” that needs to be tweaked.

Hmm… I’ll check into that. Assuming the “fudge factor” is a simple calculation from the sample rate that should be easy to fix.

[Gale_Andrews]

I would like to suggest an option for 5 decimal places of precision for the peak value, as in Sample Data Export (or just provide that with no option).


Gale

[steve]

@flynwill

Still what would be really nice would be a primitive call written in C++ (or whatever the native language of Audacity is) that efficiently went through a sound and return the sum of the samples, the sum of the samples squared, the max and the min.

In version 4 plug-ins we now have

(get '*SELECTION* 'PEAK-LEVEL)

Missing features

Audacity does have built-in functions for calculating max peak, min peak and RMS, but Nyquist cannot access arbitrary functions in Audacity. Also I think there is a slight bug in the RMS function in Audacity which I’ve been meaning to look at. We are gradually giving Nyquist more access to Audacity and vice verse, but it takes time. I’m hoping that eventually Nyquist will have access to Audacity’s scripting API, which would then allow it to use most of Audacity’s functions and commands (that would be extremely powerful).

[flynwill]

Here’s an updated version. Please give it a try.

I renamed it “ACXcheck” because it now compares the values it gets against the ACX requirements and reports if the track passes or what’s wrong if it doesn’t.

I fixed the “fudge factor”, the A-weighted RMS numbers should now be correct for sample rates from 44.1 kHz to 192 kHz, They are slightly off at 384 kHz. I could fix that by adding another point to the data if it is really used. For sample rates below 24.4 kHz the double pole in the A-filter at 12.2 kHz is probably causing problems (what does the code do with
poles that are above the nyquist frequency?). I suspect that if it is desired to fix this a filter without those poles should be substituted for the lower sample rates.

The “noise floor” values are computed from the quietest 500 mS found in the clip. This may return optimistic values if the noise has very low frequency components (like we have see in some recent posting from USB microphones) or if the noise is not constant. Still I think that for 98.8% of the cases it should give a useful indication.

Steve: Your test using white noise to compare A-weighting between sample rates is all wrong. With a higher sample rate there is a lot more energy in white noise because of the higher nyquist frequency. The fudge factor is set so that a 1 kHz tone will give the same value A-weighted and flat.

Gale: I would love to add more digits of precision to the non-dB numbers, but AFIK there is only one global format string for all floating point printing float-format, So adding digits to the linear number forces more digits on the dB number – not pretty.
acx-check.ny (4.45 KB)

[Robert_J_H]

@flynwill

Here’s an updated version. Please give it a try.

I renamed it “ACXcheck” because it now compares the values it gets against the ACX requirements and reports if the track passes or what’s wrong if it doesn’t.

I fixed the “fudge factor”, the A-weighted RMS numbers should now be correct for sample rates from 44.1 kHz to 192 kHz, They are slightly off at 384 kHz. I could fix that by adding another point to the data if it is really used. For sample rates below 24.4 kHz the double pole in the A-filter at 12.2 kHz is probably causing problems (what does the code do with
poles that are above the nyquist frequency?). I suspect that if it is desired to fix this a filter without those poles should be substituted for the lower sample rates.

The “noise floor” values are computed from the quietest 500 mS found in the clip. This may return optimistic values if the noise has very low frequency components (like we have see in some recent posting from USB microphones) or if the noise is not constant. Still I think that for 98.8% of the cases it should give a useful indication.

Steve: Your test using white noise to compare A-weighting between sample rates is all wrong. With a higher sample rate there is a lot more energy in white noise because of the higher nyquist frequency. The fudge factor is set so that a 1 kHz tone will give the same value A-weighted and flat.

Gale: I would love to add more digits of precision to the non-dB numbers, but AFIK there is only one global format string for all floating point printing float-format, So adding digits to the linear number forces more digits on the dB number – not pretty.

Change the *float-format temporarly like e.g. this:

(defun  _fp (number places) 
  (if (stringp number) number
      (progv '(*float-format*) `(,(format nil "%1.~af" places))
             (format nil  "~a" (float number)))))
;;
;; now with rounding
(defun  _fpr (number places &aux (factor (expt 10.0 places))) 
  (if (stringp number) number
      (progv '(*float-format*) `(,(format nil "%1.~af" places))
             (format nil  "~a" (/ (round (* factor number)) factor)))))
;;
(setf output "")
(dotimes (i 10 output)
   (setf output (format nil "~a~%~a" output (_fp (real-random 0 100)  i))))
;; with rounding:
;(_fpr 1.056 2)

You can change the rounding code if you prefer the dBs to be rounded towards zero.

Robert

[steve]

@flynwill

Steve: Your test using white noise to compare A-weighting between sample rates is all wrong. With a higher sample rate there is a lot more energy in white noise because of the higher nyquist frequency.

That was just a quick illustration off the top of my head. It was “incomplete”, not “all wrong”.

Here’s a more complete version:

1. Generate white noise at 44100 Hz sample rate.
2. Measure the A-weighted RMS
3. Resample to a higher sample rate
4. Measure the A-weighted RMS. It should be the same as in step 2, but it isn’t.

@flynwill

The fudge factor is set so that a 1 kHz tone will give the same value A-weighted and flat.

Yes, that’s the correct way to calculate the fudge factor. A smart solution would be to calculate the fudge factor on the fly, then you can support arbitrary sample rates. Note that IIR filters take a little while to “get going”.

[flynwill]

Attached has some bug fixes – One dealing with accuracy issues in the technique I was using to find the minimum RMS using “peak”. Second I added a 10Hz 8’th order (probably overkill) high pass filter ahead of the noise floor function to fix problems with inconsistent result from samples with changing DC offsets (The “Million Tears” sample recently posted to the audio book forum in particular.)

It also checks the clip length and will refuse to process more than 100M samples (about 40 minutes at 44.1) so as to not crash Audacity.

Steve: I believe that the attached will pass your resampled noise test with reasonable accuracy. Note that the nyquist “resample” function has a documented -0.5 dB scale to prevent clipping. Hadn’t thought about computing the “fudge factor” on the fly, I guess it wouldn’t be that hard, generate a bit of 1 kHz at the input sample rate, pass it through the filter, measure and voila! However I think the interpolated table that’s in the code is probably good enough. It’s exact at the common sample rates, and within a small fraction of a dB in between.

Robert: In a word “Yuck”. I implemented it using that technique, but it is less than elegant. How come the language comes so close to implementing “printf” and yet misses?
acx-check.ny (5.47 KB)

[Robert_J_H]

@flynwill

Robert: In a word “Yuck”. I implemented it using that technique, but it is less than elegant. How come the language comes so close to implementing “printf” and yet misses?

I see, we do not share the same thinking about elegancy.
Your code looks a bit less elegant due to the absolute digit settings and the two PROGVs needed therefore.
It would be nicer as a “With-format” macro.

You’re right, there’s a lot missing from the C printf, most notably number formats and bin/hex/oct display.

Robert

[kozikowski]

How can this still be version 1?
Koz

[kozikowski]

We got the same numbers on the Assassination Vacation clip. Your noise floor was lower than mine, we assume because you scanned for a lower value than I did. Yes, the clip does exceed the -3.0 peak threshold, but since I got the clip from an actual audiobook, that was probably OK. I wouldn’t be shocked if the original WAV files were compliant and the conversion to MP3 caused the error.

But you’re right. That would normally have been flagged.

I suspect the name “Sarah Vowell” on the submission is enough to halt flagging for minor transgressions.

I used version 1 of the software.

Koz

[kozikowski]

I analyzed one of the “Eden Green” clips and again you came up with less noise than I did, but you also flagged a peak of -3 as a violation (attached).

-3.0dB, -20.4dB, -69.4dB (classic method)

Koz

[flynwill]

Which “Eden Green” clip? (I’ve got several but all fail for too high a noise floor).

Technically -3db is 0.0707946, and the clip is clearly reporting 0.709961, so the clip does exceed the spec (but by less than 0.05 dB thus it still says “-3.0”. )
Not sure what the best fix would be… moving the “fail” threshold to -2.95 dB seems like it would just cause occasional false hope. Changing the rounding
on the db display so that -2.99999 dB displays as -2.9 dB might be an option, but it rails against my scientific instincts.

That it should find a somewhat lower noise floor than the “classic method” isn’t too surprising as it is searching for the quietest 0.5 sec in the clip. However there are a number of issues with the technique I’m using to find that “non-peak” and they may well still be corner cases that don’t work properly. Email me the test clip.

Sorry about the “version”… Next release it will be different.

[kozikowski]

My objection is more about the display than anything else. The two ends of the INFO panel appear to not agree with each other. I need to email or post it on KozCo. It’s too long for the forum.
Koz

[flynwill]

Offline Koz was inquiring about making the ACX pass/fail information easier to pick out from the rest.

I searched the documentation and came up empty but figured I’d inquire with the wizards here: Is there a mechanism to control font, text color, or style (ie bold) in the printout from a plugin? It would be nice to be able to put up the “failed” information in red for example.

[steve]

@flynwill

Is there a mechanism to control font, text color, or style (ie bold) in the printout from a plugin? It would be nice to be able to put up the “failed” information in red for example.

XXXXXX      X       XXX   XX
XX         X X       X    XX
XX        X   X      X    XX
XXXXX    XXXXXXX     X    XX
XX      XX     XX    X    XX
XX     XX       XX  XXX   XXXXXX


Length of selection: 59.322 seconds.
2616101 samaples at 44100 H.
Mono track.

DC offset: 0.000019%
Peak Level 0.709961 (-3.0 dB)
....
....

[flynwill]

Oh that is sooo 1970’s

[steve]

The proper answer:

Text output from Nyquist plug-ins is just plain text, or to be more precise, the message box that Audacity creates to display string data returned from a Nyquist plug-in only displays plain text characters. So no colour, just black text.

[flynwill]

So I’m zeroing in on calling this plugin “done” for now. It looks like the “proper” home for plugins that are ready for prime time is the plugins page on the wiki http://wiki.audacityteam.org/wiki/Nyquist_Analyze_Plug-ins and it’s parent http://wiki.audacityteam.org/wiki/Download_Nyquist_Plug-ins.

So it would appear that all that is needed is to upload the plugin to the wiki and the edit those two pages to include it in the manner of the other plugins there.

Am I missing anything?

[steve]

@flynwill

So it would appear that all that is needed is to upload the plugin to the wiki and the edit those two pages to include it in the manner of the other plugins there.

Am I missing anything?

I’ll not have time to do that in the next few days, but I’ll be happy to upload it to the wiki later in the week.
Something you could do to help would be to start a new topic for your ACX version to include your “final” version of the plug-in and documentation in the style of other plug-ins on the wiki.

Something I was looking at doing was to make a version that creates labels, so that it can highlight specific parts of a track that may need attention. For example, if the peak level exceeds -3 dB in a couple of places, it labels those places. Perhaps that’s something that you would be interested in having a look at developing?

[flynwill]

@steve

@flynwill

So it would appear that all that is needed is to upload the plugin to the wiki and the edit those two pages to include it in the manner of the other plugins there.

Am I missing anything?

I’ll not have time to do that in the next few days, but I’ll be happy to upload it to the wiki later in the week.

It’ll probably be at least a week before it’s ready anyway. I’ll give it to koz to beta test first. It would appear that as a lowly forum user I can update the wiki myself. Since it should be easy enough to use copy/paste to match the formatting given to the other plugins, it would seem I should be able to just do this myself.

Something you could do to help would be to start a new topic for your ACX version to include your “final” version of the plug-in and documentation in the style of other plug-ins on the wiki.

Indeed I can do that as well.

Something I was looking at doing was to make a version that creates labels, so that it can highlight specific parts of a track that may need attention. For example, if the peak level exceeds -3 dB in a couple of places, it labels those places. Perhaps that’s something that you would be interested in having a look at developing?

That could be interesting, it would also be interesting to mark the region identified as “noise floor”. Both use snd-maxsamp to get the value of the peak or minimum, but snd-maxsamp doesn’t return a location, is there a similar function that does?