"Wave Stats" plug-in

OK, thanks! I used wave-info.ny posted here https://forum.audacityteam.org/t/firebox-or-inspire/174/1

So to get the average RMS for a two minute recording I guess I can copy the file, calculate 30 seconds, delete the 30 seconds, calculate the next 30 seconds, delete and so on until the end.

Interesting pdf! I have worked in broadcasting many years but never thought of that part of EBUs work.

Pretty sweet pluggin, thanks.

I did run the pluggin, and there is one minor aspect that I do not understand, as well as one improvement suggestion.

  • I generated a full scale sine wave (generate, tone, sine, frequency 1000, amplitude 1). I process 10s of audio samples through the wave stats pluggin.
    Results reported:
    peaks all at 0dBFS as expected
    DC 0% also expected
    RMS value at -3dBFS, okay
    RMS (A-Weighted) at -6.6dBFS ???, this I do not understand.
    A-Weighting is supposed to be precisely centered around 1K, so why 3.3dB less after A filtering ?
    I checked the spectrum of the generated signal, it looked relatively clean.

  • improvement: it would be sooooo conveninent to just be able to copy and paste the output of the analysis :stuck_out_tongue:

Some more on this:

  • I redid the experiment sucessfully when using 44100Hz. RMS Aw reads -3dBFS sharp.
  • The root cause is that I was running the previous project under 8000KHz. Reading the frequency analysis, the sine is then not so clean. I guess probably an issue with the tone generation (this has been a bug for a while on Audition also…)

On windows, the whole window can be copied as text by simply pressing Ctrl-c, as soon as the dialog appears.
The result looks like this:

---------------------------
Nyquist
---------------------------
Length of selection: 521.790 seconds.
23010939 samples at 44100 Hz.
Analysis of first 521.790 seconds:
(23010939 samples)

CHANNEL 1
Peak Level: -1.9 dBFS
Peak Positive: -1.9 dBFS
Peak Negative: -1.9 dBFS
DC offset: 0.0 %
RMS: -21.7 dBFS
RMS (A-weighted): -28.5 dBFS

CHANNEL 2
Peak Level: -2.0 dBFS
Peak Positive: -2.0 dBFS
Peak Negative: -2.1 dBFS
DC offset: 0.0 %
RMS: -21.5 dBFS
RMS (A-weighted): -28.8 dBFS

---------------------------
OK   
---------------------------

Please note that it isn’t quite the same plug-in as my version has no thirty s limit, thus the odd selection length.

Thanks Robert, works like a charm.
Windows raises a warning beep somewhat telling I’m doing something not allowed, but it works still, thanks.

Back to your version of the pluggin: How do you manage time then : you have RMS windows and perform RMS averaging in the end?
(side question: where can I pick this from? )

I don’t use windowing. That’s actually only needed for real time display, e.g. display update at each second.
This means, all samples are summed and averaged.
I might go back to windowing when I want to exclude silences as it is e.g. done in the R128 recommendation.
However, the difference is not that overwhelming.
I’m sure that Steve will soon enable longer durations since Acx/Audible measure 60 s blocks for instance.
Until then:
wavestats.ny (2.3 KB)

Thanks Roberts!
So you make a real full length RMS then?
as in here:

Pretty cool then.
Note time windowing for RMS has other use than real time display, it can provide interesting stas on a signal, such as min RMS, max, average, and total (mostly identical to avg).
Note Audition gives you also the A-Weighted RMS, which happens to be very usefull when you deal with some DC, but I agree here this is just a nice to have.

Me again.
I think I found a minor bug in your pluggin.
When measuring a Signal that is always positive due to DC Offset, I find the peak negative value is wrongly initialized:
(the same problem projects to peak positive value if you invert the signal)
The problem is also present in the original pluggin.

---------------------------
Nyquist
---------------------------
Length of selection: 1.055 seconds.
8441 samples at 8000 Hz.
Analysis of first 1.055 seconds:
(8441 samples)

Mono Track.
Peak Level: -20.9 dBFS
Peak Positive: -20.9 dBFS
Peak Negative: -1.$ dBFS
DC offset: 8.9 %
RMS: -21.0 dBFS
RMS (A-weighted): -63.4 dBFS

---------------------------
OK   
---------------------------

“Absolute silence” is “-infinity” dB. (minus infinity)
What number is less than “- infinity”?

The way that “- infinity” is represented by computers is different on different systems. On Linux it is represented as “-inf”, on your system it is represented as “-1.$”.

They say about computers “garbage in; garbage out”, or to put that another way, if you input invalid data, you shouldn’t expect to get something sensible out. :wink:
In your test, the negative going peaks are on the wrong side of silence, that is, they are less than - infinity.

There are an infinite number of odd numbers.
There are an infinite number of even numbers.
How many numbers are there? (odd + even) ?

I don’t think that additional Rms values, based on windowing give useful information.
The min Rms value is often -infinity, depending on the window length.
There’s not much agreement on how to define those window lengths and how to treat “silence”.
Minima and maxima approach the average the longer the window length gets (until the entire track length is reached).
R128 differentiates 4 Window length, momentary Rms (some centiseconds), short-term (1 s), long-term (3 s) and integrated Rms (entire audio).
It is in principle no problem to display avg, max and min for those types.
However, the meaningfulness is rather questionable.

It is better to implement statistical functions that are independant, such as mean + Standard deviation (e.g. in percent).
Unfortunately, this values are actually only applicable to a normal distribution (Gauss) and that’s not always given–only for white noise in fact.

I’m writing a plug-in that shows instead the whole Rms histogram (of course resized to fit into the dialog box, i.e. 25 lines is the max).
This represents the energy distribution much better in my opinion.

For example, you can see at a glance at the -inf (= 0) bar how high the silence percentage is, and if there’s nothing but instead at around -73 dB, the audio has been dithered and that’s the silence amount now.
There will be other peaks too, e.g. for different speakers in a phone conversation or a noisy input.

By the way, Steve’s plug-in gives the A-weighted Rms too.
Your sample-output shows that the Rms level is corrected by -40 dB due to the heavy DC-offset.

Waking up an old thread…

I just finished a new cut on the “wave stats” plugin. The attached is roughly based on “measurements.ny” however it’s been reworked to remove the time limitations that measurements.ny had (it would fail if you selected more than about 45 seconds) stats.ny has a similar limitation, but it forces users to not analyze more than 30 seconds.

It needs a better name, for the moment I just called it “my Amplitude Statistics”.

I’ve tested it on ~40 minute stereo files, and it can give you the answer in about 30-40 seconds. However the progress bar is totally confused by the multiple passes that the code takes.
better-measurement.ny (2.72 KB)

I think it does that during auto analysis of multiple tracks, too. It never failed to give numbers and the numbers seemed rational, so I didn’t worry about it.

Koz

Nice one flynwill, good to see this resurrected.

Unfortunately if you go much bigger than that, Audacity will crash.
The problem is snd-maxsamp which computes the samples in memory and retains them until the script completes. As Nyquist is purely 32-bit, I think that gives you a limit of 2 GB before it explodes. This problem is not unique to your plugin, it affect many (most?) Nyquist plugins and there is no way around it until we improve the way that Nyquist accesses the audio.

Probably better to use PEAK and set a reasonable limit.

I think what is confusing the progress bar is the time taken to load and release memory. The actual processing is very fast, and that is shown reasonably accurately by the progress bar, but writing hundreds of MB of data to ram, (and then releasing the ram), takes time that the progress indicator cannot take account of.

There is another problem which is that A-weighting filter is specific to 44.1 kHz sample rate. If, for example, you analyze white noise with a peak of 1.0, the A-weighted result shows -7.1 dB, which I think is about right, but for a sample rate of 192 kHz it shows -12.85 dB. Fortunately the Nyquist filters are clever enough to recalculate their parameters so that the “shape” of the filter remains correct, so it’s only the “fudge factor” that needs to be tweaked.

Yes I figured there might be a limitation there somewhere.

This code uses iterated snd-avg calls to produce the average and get around the max block-size limitation that the previous version had. I did try writing the brute force loop through all the sound sample and accumulate s, s^2, smax and smin but that was going to be a LOT slower than the iterated snd-avg functions.

I also worry a bit, there may be an accuracy problem still. If you naively add up 2^20 samples in 32-bit floating point hoping to generate an average, toward the end of the sum you may be discarding a lot of bits from your samples. I don’t know how snd-avg is implemented – do you know if it has anything that might mitigate this source of error?

Still what would be really nice would be a primitive call written in C++ (or whatever the native language of Audacity is) that efficiently went through a sound and return the sum of the samples, the sum of the samples squared, the max and the min.

[quote=“flynwill”]
However the progress bar is totally confused by the multiple passes that the code takes.
[/quote]

I think what is confusing the progress bar is the time taken to load and release memory. The actual processing is very fast, and that is shown reasonably accurately by the progress bar, but writing hundreds of MB of data to ram, (and then releasing the ram), takes time that the progress indicator cannot take account of.

I found a brief discussion of next cut at plugins being able to provide feedback to the progress bar to make it better… did that ever go anywhere?

There is another problem which is that A-weighting filter is specific to 44.1 kHz sample rate. If, for example, you analyze white noise with a peak of 1.0, the A-weighted result shows -7.1 dB, which I think is about right, but for a sample rate of 192 kHz it shows -12.85 dB. Fortunately the Nyquist filters are clever enough to recalculate their parameters so that the “shape” of the filter remains correct, so it’s only the “fudge factor” that needs to be tweaked.

Hmm… I’ll check into that. Assuming the “fudge factor” is a simple calculation from the sample rate that should be easy to fix.

I would like to suggest an option for 5 decimal places of precision for the peak value, as in Sample Data Export (or just provide that with no option).


Gale

In version 4 plug-ins we now have

(get '*SELECTION* 'PEAK-LEVEL)

Audacity does have built-in functions for calculating max peak, min peak and RMS, but Nyquist cannot access arbitrary functions in Audacity. Also I think there is a slight bug in the RMS function in Audacity which I’ve been meaning to look at. We are gradually giving Nyquist more access to Audacity and vice verse, but it takes time. I’m hoping that eventually Nyquist will have access to Audacity’s scripting API, which would then allow it to use most of Audacity’s functions and commands (that would be extremely powerful).

Here’s an updated version. Please give it a try.

I renamed it “ACXcheck” because it now compares the values it gets against the ACX requirements and reports if the track passes or what’s wrong if it doesn’t.

I fixed the “fudge factor”, the A-weighted RMS numbers should now be correct for sample rates from 44.1 kHz to 192 kHz, They are slightly off at 384 kHz. I could fix that by adding another point to the data if it is really used. For sample rates below 24.4 kHz the double pole in the A-filter at 12.2 kHz is probably causing problems (what does the code do with
poles that are above the nyquist frequency?). I suspect that if it is desired to fix this a filter without those poles should be substituted for the lower sample rates.

The “noise floor” values are computed from the quietest 500 mS found in the clip. This may return optimistic values if the noise has very low frequency components (like we have see in some recent posting from USB microphones) or if the noise is not constant. Still I think that for 98.8% of the cases it should give a useful indication.

Steve: Your test using white noise to compare A-weighting between sample rates is all wrong. With a higher sample rate there is a lot more energy in white noise because of the higher nyquist frequency. The fudge factor is set so that a 1 kHz tone will give the same value A-weighted and flat.

Gale: I would love to add more digits of precision to the non-dB numbers, but AFIK there is only one global format string for all floating point printing float-format, So adding digits to the linear number forces more digits on the dB number – not pretty.
acx-check.ny (4.45 KB)

Change the *float-format temporarly like e.g. this:

(defun  _fp (number places) 
  (if (stringp number) number
      (progv '(*float-format*) `(,(format nil "%1.~af" places))
             (format nil  "~a" (float number)))))
;;
;; now with rounding
(defun  _fpr (number places &aux (factor (expt 10.0 places))) 
  (if (stringp number) number
      (progv '(*float-format*) `(,(format nil "%1.~af" places))
             (format nil  "~a" (/ (round (* factor number)) factor)))))
;;
(setf output "")
(dotimes (i 10 output)
   (setf output (format nil "~a~%~a" output (_fp (real-random 0 100)  i))))
;; with rounding:
;(_fpr 1.056 2)

You can change the rounding code if you prefer the dBs to be rounded towards zero.

Robert

That was just a quick illustration off the top of my head. It was “incomplete”, not “all wrong”.

Here’s a more complete version:

  1. Generate white noise at 44100 Hz sample rate.
  2. Measure the A-weighted RMS
  3. Resample to a higher sample rate
  4. Measure the A-weighted RMS. It should be the same as in step 2, but it isn’t.

Yes, that’s the correct way to calculate the fudge factor. A smart solution would be to calculate the fudge factor on the fly, then you can support arbitrary sample rates. Note that IIR filters take a little while to “get going”.

Attached has some bug fixes – One dealing with accuracy issues in the technique I was using to find the minimum RMS using “peak”. Second I added a 10Hz 8’th order (probably overkill) high pass filter ahead of the noise floor function to fix problems with inconsistent result from samples with changing DC offsets (The “Million Tears” sample recently posted to the audio book forum in particular.)

It also checks the clip length and will refuse to process more than 100M samples (about 40 minutes at 44.1) so as to not crash Audacity.

Steve: I believe that the attached will pass your resampled noise test with reasonable accuracy. Note that the nyquist “resample” function has a documented -0.5 dB scale to prevent clipping. Hadn’t thought about computing the “fudge factor” on the fly, I guess it wouldn’t be that hard, generate a bit of 1 kHz at the input sample rate, pass it through the filter, measure and voila! However I think the interpolated table that’s in the code is probably good enough. It’s exact at the common sample rates, and within a small fraction of a dB in between.

Robert: In a word “Yuck”. I implemented it using that technique, but it is less than elegant. How come the language comes so close to implementing “printf” and yet misses?
acx-check.ny (5.47 KB)