ACX Check fails in 2.4.1

Mac Mini. OSX 10.9.5 Mavericks. 16GB memory. 250GB SSD, 41GB left.

ACX Check fails under Audacity 2.4.1

Launch Audacity. Generate > Noise, Pink, 0.9, 30 min

Select the whole track.

Analyze > ACX Check > ACX Check seems to launch and I get a blue progress bar. Pause. Spinning Beachball, Sometimes second spinning beachball, ACX Check vanishes.

========================

ACX Check succeeds under Audacity 2.3.3

Launch Audacity. Generate > Noise, Pink, 0.9, 30 min

Select the whole track.

Analyze > ACX Check > ACX Check seems to launch and I get a blue progress bar. Pause. Sometimes Spinning Beachball, ACX Check INFO panel presents.

Screen Shot 2020-05-23 at 19.05.44.png
OK the INFO panel. Spinning beachball and then the ACX Check progress panel vanishes.

Repeat as needed.

That’s it. That’s as far as I got.

I don’t have a Catalina machine.

Koz

I can repeat this in 10.13 High Sierra…later.

Koz

This one. I don’t know of another release.

;nyquist plug-in
;version 3
;type analyze
;name “ACX Check”
;action “Compute statistics about clip and compare to ACX requirements”
;author “Will McCown”

;
; 2015-07
; release 1.0

Koz

High Sierra 10.13 same behavior. ACX Check succeeds under Audacity 2.3.3 and not under 2.4.1.

Koz

Here’s the error:

nyx_error returned from ACX Check.
The maximum number of sample blocks has been
reached, so audio computation must be terminated.
Probably, your program should not be retaining
so many samples in memory. You can get and set
the maximum using SND-SET-MAX-AUDIO-MEM.
error: audio memory exhausted
Function: #<Subr-SND-LENGTH: #55b584dd48d0>
Arguments:
  #<Sound: #55b5b868b4b8>
  100000000
Function: #<FSubr-SETQ: #55b584dd7d20>
Arguments:
  S-LEN
  (SND-LENGTH S-RMS MAX-LEN)

and here’s the problem:

(setq max-len 100000000)  ; Empirically determined on my machine, may be hardware dependent.

It appears that not only is hardware dependent, but also Nyquist version dependent.

The easiest “fix” is to remove a zero in the above. This will cause the plug-in to throw an error if more than 10 million samples are selected (rather than 100 million).

 "Selection too long for analysis, please select shorter section~%"

Here’s what the Nyquist manual says about SND-SET-MAX-AUDIO-MEM (emphasis mine)@

snd-set-max-audio-mem(bytes) [SAL]
(snd-set-max-audio-mem bytes) [LISP]

>
> Sets a limit (a FIXNUM in bytes) on the amount of main memory that Nyquist will allocate for SOUNDs (this does not count table memory). The default is about 1GB. The return value is the previous limit, in bytes. This is not a limit on how big sounds can be. Since Nyquist computes sound incrementally, it can generally play sounds or write sounds to files without storing them in memory. However, > **if you store large sounds in variables, memory usage can exceed your available RAM, causing extremely slow computation as main memory is swapped to and from disk or Nyquist might run out of memory and crash.** > The goal of placing a limit on audio memory is to terminate computations before memory is totally exhausted, allowing Nyquist to print an error message and allowing the user to view or save work. Generally, if you see the message "The maximum number of sample blocks has been reached, ...," > **_[u]you should fix your code to avoid accumulating samples in memory[/u]_**> , e.g. do not assign sounds to global variables. Alternatively, you can use this function to increase the limit, but of course you are still limited by the actual size of memory on your computer, and exceeding that could cause severe performance problems in Nyquist and and any other applications that are running. See also Section "Command Line" for command line options to limit yquist memory, run time, and file access (these options are not available through the NyquistIDE.)

Prior to Nyquist being updated, we relied on plug-in writers to set a limit in cases where large selections could be held in memory. Will McCown did so in ACX Check, and chose a fairly ambitious maximum based on tests on his machine. Unfortunately the "maximum number of samples" that he chose is so large that Nyquist reaches it's maximum audio memory before the maximum number of samples is reached.

So one solution is to reduce the maximum number of samples (as in the "easiest fix"), and another option is to increase the maximum allowed audio memory. This can be done by adding this line somewhere near the top of the code:

```text
(SND-SET-MAX-AUDIO-MEM 3000000000)

The number (3 billion bytes) is sufficient for this plug-in to work with 100 million samples, though if the computer has less than 3 billion bytes of free RAM, then the plug-in is very likely to crash and take Audacity down with it.


So what’s the best fix?
As it says in the Nyquist manual, the best fix would be to rewrite the plug-in to avoid accumulating samples in memory.

Unfortunately, the problem part of the code that accumulates samples in memory is Will McCown’s rather strange code for detecting the noise floor.

I’ve added a warning of this problem on the ACX Check download page:
This plug-in may fail without warning on selections over 10 million samples duration.

And how long is that piece of string in real money ? :confused:

Peter.

Divide by the sample rate.

10000000 / 11025 = 907 seconds = about 15 minutes.
10000000 / 44100 = 226.76 seconds = about 4 minutes.
10000000 / 96000 = 104 seconds = about 2 minutes

These are all on the safe side - you will probably get a fair bit more before the problem appears.

So that explains the complaints of 2-4 minute erratic behavior. Why did earlier Audacity versions appear to succeed?

ACX published restrictions are 120 minutes and they used to have a filesize restriction I can’t find. So the easy answer is roll Audacity back to 2.3.3 for audiobooks. ACX Check is the hero app.

Where is 2.3.3? for those keen to read for audiobooks, make a fortune, and retire to the cottage in Coq au Vin.

Koz

Earlier versions of Audacity had an earlier version of the Nyquist library.

In the earlier version of Nyquist, the plug-in would succeed up to 100 million samples provided that the computer has enough free memory. If the computer did not have enough free RAM then the plug-in and Audacity would probably crash.


It’s available on FossHub. Click on the “old versions” tab below the main file list.
but given the crash risk, a better solution would be to use effects that don’t have this problem.

As you’re probably aware, I’ve never been much of a fan of the noise floor measurement in this plug-in. Personally I’d prefer to use the Contrast effect to measure the noise floor.

https://www.kozco.com/tech/audacity/ACXTesting/ACXTesting.html

That will be popular.

Koz

“Contrast” now works with mono or stereo (has done for a few years now).

If you needed a very reliable measurement of the noise floor in a recording, how would you do it? (any tools / any software is allowed)

“Contrast” now works with mono or stereo (has done for a few years now).

Yes there are patches to that page. That’s one reason it’s not linked anymore; it’s in a resting phase. But that process is the reason ACX Check exists.

“How can I be sure I’m passing ACX Technical Standards?”

Well, (rolling up sleeves)…

I guess I can buff it up and bring it out of its resting phase.

I can see what Flynwill is doing. Last I checked, they were chowing down on the artichokes he was growing in his back yard/garden.

I favor weeds.

Koz

OK, that’s reasonable, though perhaps not possible with absolute certainty as we don’t have access to their robot.
What we can do is to have our own “robot” (“ACX-Check”) that emulates their robot as closely as possible given the information available.

So long as we ignore long silences (which should normally only occur at the start / end of chapters), then we can easily get an accurate value for RMS. (The long lead-in silence will throw off the accuracy a bit if it is included in the test).

Measuring the peak is not a problem.

Measuring the noise floor is the biggest problem. We know that the current ACX-Check.ny has never handled this very robustly, and now we have a new problem with it in Audacity 2.4.x.

I can quite easily add that command “SND-SET-MAX-AUDIO-MEM” so that ACX-Check.ny will behave as it did in older versions of Audacity, but:

  1. It will still be as unreliable as the old version.
  2. It still risks crashing if not enough RAM is available,

so back to my question:
“If you needed a very reliable measurement of the noise floor in a recording, how would you do it? (any tools / any software is allowed)”

we don’t have access to their robot.

No, but their published standards are clear both loudness and noise is measured in RMS.

so back to my question:
“If you needed a very reliable measurement of the noise floor in a recording, how would you do it? (any tools / any software is allowed)”

Contrast is fine with me. The goal is semi-reasonable and rapid testing, not counting how many angels can dance in the area under a curve.

A bonus if voice volume measurement ignored long stretches of silence.

Release 1.1?

Last time someone paid me to do this I had to average the needle wiggles on a Hewlett-Packard audio meter. This is better.

Koz

An alternative is the “Measure RMS” plug-in, which is shipped with Audacity and just needs to be enabled in the Plug-in Manager: Measure RMS - Audacity Manual


I guess that ideally we should see the level drop below 60 dB RMS at least a couple of times per minute throughout the track, as well as at the start (?)

just needs to be enabled

Don’t wander off the goal. Any tool you don’t have to think about or take any action.

least a couple of times per minute

I don’t think getting silences like that is likely. That would mean the performer would have to stop breathing, not good in a theatrical performance.

We should guard against false negatives. There’s no guarantee that the reader is going to measure a complete chapter, so there’s no guarantee of two Room Tones in the piece. Or even one.

There’s always going to be a difference between our robot and theirs. Theirs is always going to have room tone under pain of rejection. That also gives us the Standard Forum Test Clip. Guaranteed all three performance samples.

I don’t see any way around the way it works now, with publication of known shortcomings. I do think it should be published that the tool is looking for “about 3/4 second” of background sound in place of the half-second which it’s actually using. That eliminates the User Race: “It’s half second. No it’s not. Yes it is.”

Koz

Are we at ACX Check Release 1.1?

Koz

Here’s an actual audio example that was accepted by ACX.

Note that the highlighted section has a higher noise floor than the rest of the track. In this section the noise floor is fractionally over the -60 dB specification. The actual noise floor during this minute is around -58 dB. The rest of the track is well under at around -70 dB. I suspect that the author got away with this because either:

  1. The robot missed it. Perhaps it only tests the lead-in silence, and leaves other noise analysis to the human check.
  2. The robot flagged it as “nearly OK” and a human then reviewed passed it.
  3. The robot gives multiple pass / fail results for a track, such as a pass / fail for each minute.
    ✓✓✗✓✓✓✓✓✓✓✓✓…
    Ideally the result should be all passes. If nearly all passes, a human may review and choose to override or agree with the robot.

I doubt that the robot would do (b).
I think (a) or (c) are possible.

Do we want “our” robot to be able to detect problems like this?