Code to Select Audio

steve · August 30, 2019, 5:21pm

Yes

This was a simplified example, to illustrate how you can hang onto the first RMS signal while you get the other channel’s RMS signal.
The left channel’s RMS is assigned to the variable “rms-l” and the right channel’s RMS is assigned to the variable “rms-r”.
Once you have both of those values assigned, you can use them whatever way you like. I just took a simple average of the two.
To get the true RMS of the two channels would be like this:

(setf mean-sq-left (mult rms-l rms-l))
(setf mean-sq-right (mult rms-r rms-r))
(setf mean-sq-both (mult 0.5 (sum mean-sq-left mean-sq-right))
(setf rms-both (snd-sqrt mean-sq-both))

HBB.ThinkTank · August 30, 2019, 8:56pm

Wow, Steve, it looks like this should do the trick:

;version 4

(setq FMT-db "%#.1f")
(setq FMT-val "%#.3f")

(defun fmt-pretty (dformat &rest args)
  (prog2 
    (setf *float-format* dformat)
    (apply #'format args)
    (setf *float-format* "%g")))


(setf myrms (let ((rms (if (arrayp *track*)
    (progn ;starting a block of code
      (setf rms-l (rms (aref *track* 0) 4))
      (setf rms-r (rms (aref *track* 1) 4))
	  (setf mean-sq-left (mult rms-l rms-l))
	  (setf mean-sq-right (mult rms-r rms-r))
	  (setf mean-sq-both (mult 0.5 (sum mean-sq-left mean-sq-right)))
	  (snd-sqrt mean-sq-both))	  ;return the RMS of the stereo sound
    (rms *track* 4)))
      (highest 0))
  (do ((val (snd-fetch rms)(snd-fetch rms)))
      ((not val) (linear-to-db highest))
    (setf highest (max highest val)))))

(setf myrms2 (db-to-linear myrms))

(fmt-pretty FMT-db nil "~a" myrms)
(fmt-pretty FMT-val nil "~a" myrms2)

(format nil "Hightest RMS is ~a (~a dB)" myrms2 myrms)

The only thing in this part is; I seem to be unable to properly display the results. The fmt-pretty function don’t seem to work.

While your idea works…

;version 4

(setf *float-format* "%.3f")  ;3 decimal places

(setf myrms (let ((rms (if (arrayp *track*)
    (progn ;starting a block of code
      (setf rms-l (rms (aref *track* 0) 4))
      (setf rms-r (rms (aref *track* 1) 4))
	  (setf mean-sq-left (mult rms-l rms-l))
	  (setf mean-sq-right (mult rms-r rms-r))
	  (setf mean-sq-both (mult 0.5 (sum mean-sq-left mean-sq-right)))
	  (snd-sqrt mean-sq-both))	  ;return the RMS of the stereo sound
    (rms *track* 4)))
      (highest 0))
  (do ((val (snd-fetch rms)(snd-fetch rms)))
      ((not val) (linear-to-db highest))
    (setf highest (max highest val)))))

(setf myrms2 (db-to-linear myrms))

(format nil "Hightest RMS is ~a (~a dB)" myrms2 myrms)

…it seems not to allow me to display the result in different formats (say “%.1f” for dB and “%.3f” for linear).

Next step will be the highest RMS of a selection of several tracks (like an album).

HBB.ThinkTank · August 30, 2019, 10:53pm

I can’t say whether his code will not produce an error at some point but so far it worked:

;version 4

(setf highestrms (get '*SCRATCH* 'hbb-album-highestrms))
(unless (numberp highestrms) (setf highestrms 0))

(setq FMT-db "%#.1f")
(setq FMT-val "%#.3f")

(defun fmt-pretty (dformat &rest args)
  (prog2 
    (setf *float-format* dformat)
    (apply #'format args)
    (setf *float-format* "%g")))


(setf myrms (let ((rms (if (arrayp *track*)
    (progn ;starting a block of code
      (setf rms-l (rms (aref *track* 0) 4))
      (setf rms-r (rms (aref *track* 1) 4))
	  (setf mean-sq-left (mult rms-l rms-l))
	  (setf mean-sq-right (mult rms-r rms-r))
	  (setf mean-sq-both (mult 0.5 (sum mean-sq-left mean-sq-right)))
	  (snd-sqrt mean-sq-both))	  ;return the average of the two RMS signals
    (rms *track* 4)))
      (highest 0))
  (do ((val (snd-fetch rms)(snd-fetch rms)))
      ((not val) (linear-to-db highest))
    (setf highest (max highest val)))))

(setf highestrms (max highestrms (db-to-linear myrms)))
(putprop '*SCRATCH* highestrms 'hbb-album-highestrms)

(setf highestrms2 (linear-to-db highestrms))

(fmt-pretty FMT-db nil "~a" highestrms2)
(fmt-pretty FMT-val nil "~a" highestrms)

(format nil "Hightest RMS is ~a (~a dB)" highestrms highestrms2)

I tested it so far on two tracks, the first one a stereo, the second one a mono, where individually calculated the stereo track had the higher RMS, the second one the lower RMS. When I calculated the RMS for both tracks, it returned both times the RMS of the first track.

As suggested in the SCRATCH Tutorial, I’ll need a second Nyquist effect to delete hbb-album-highestrms:

(remprop '*SCRATCH* 'hbb-album-highestrms)

Now let’s get back to the beginning.

In rms track 4, the 4 stands for 4 Hz. I think the time window will be the same for every sound, i.e. it is independent of the number of samples. I guess if I wanted to increase the time window to 10 seconds, the code would be rms track 0.1? It does work and since the time window is longer the RMS is lower.

To complete this little exercise (not counting the format functions) I should add the peak value. This is the basic code:

;version 4

(snd-maxsamp *track*)

Edit: snd-maxsamp requires also an arrayp for stereo sound.

HBB.ThinkTank · August 30, 2019, 11:08pm

I still have the feeling that this code does first create the RMS for the whole left channel, then the RMS for the whole right channel and then calculates the average RMS?

If so, how could you get the RMS for the left channel of the first time window, then the RMS of the right channel of this time window and then calculate the average RMS for this first time window, and then (and only then) procede the the next time window, now comparing the average RMS and retain the higher RMS?

Edit: Found the answer to that by testing, see next post

HBB.ThinkTank · August 31, 2019, 1:14pm

Now I have run a long test series and no: it seems to get the stereo-RMS for each time window and store the RMS for the time window with the highest RMS. Meaning that this code does exactly what I want it to do! Thank you, Steve, for guiding me though the process.

Next I’ll add maximum peak and the crest factor. Friedemann Tischmeyer’s DR-Meter seems to use a somewhat arbitrary definition of the dynamic range.

HBB.ThinkTank · August 31, 2019, 2:34pm

Now this is my final code (for now - though I have some ideas for the future):

;nyquist plug-in
;version 4
;type analyze
;name "Measure Highest RMS"
;action "Measuring RMS level of the loudest section"
;info "By Holger Burghardt under GPL v3"
;author "Holger Burghardt / HBB Think Tank"
;release 1.0.0
;copyright "Released under terms of the GNU General Public License version 3"

;; Released under terms of the GNU General Public License version 3:
;; https://www.gnu.org/licenses/gpl-3.0.html
;;
;; For information about writing and modifying Nyquist plug-ins:
;; https://wiki.audacityteam.org/wiki/Nyquist_Plug-ins_Reference

;; This plugin uses code from Audacity’s Nyquist forum:
;; https://forum.audacityteam.org/t/code-to-select-audio/53816/1 (primarily)
;; https://forum.audacityteam.org/t/acx-check-help-to-understand-its-code/53759/1 (to a much lesser extent)
;; Special thanks to Steve Daulton


(setf highestrms (get '*SCRATCH* 'hbb-album-highestrms))
(unless (numberp highestrms) (setf highestrms 0))
(setf highestpeak (get '*SCRATCH* 'hbb-album-highestpeak))
(unless (numberp highestpeak) (setf highestpeak 0))

(setq FMT-db "%#.1f")
(setq FMT-val "%#.3f")

(defun fmt-pretty (dformat &rest args)
  (prog2 
    (setf *float-format* dformat)
    (apply #'format args)
    (setf *float-format* "%g")))

(setf mypeak (if (arrayp *track*)
  (progn ;starting a block of code
    (setf peak-l (snd-maxsamp (aref *track* 0)))
    (setf peak-r (snd-maxsamp (aref *track* 1)))
    (mult 0.5 (sum peak-l peak-r))) ;return the average of the two RMS signals
  (snd-maxsamp *track*)))
(setf highestpeak (max highestpeak mypeak))
(putprop '*SCRATCH* highestpeak 'hbb-album-highestpeak)


(setf myrms (let ((rms (if (arrayp *track*)
  (progn ;starting a block of code
    (setf rms-l (rms (aref *track* 0) 4))
    (setf rms-r (rms (aref *track* 1) 4))
	(setf mean-sq-left (mult rms-l rms-l))
	(setf mean-sq-right (mult rms-r rms-r))
	(setf mean-sq-both (mult 0.5 (sum mean-sq-left mean-sq-right)))
    (snd-sqrt mean-sq-both))	  ;return the average of the two RMS signals
  (rms *track* 4)))
  (highest 0))
  (do ((val (snd-fetch rms)(snd-fetch rms)))
    ((not val) (linear-to-db highest))
    (setf highest (max highest val)))))

(setf highestrms (max highestrms (db-to-linear myrms)))
(putprop '*SCRATCH* highestrms 'hbb-album-highestrms)

(setf highestrms2 (linear-to-db highestrms))
(setf crest-highestrms (round (linear-to-db(/ highestpeak highestrms))))

(fmt-pretty FMT-db nil "~a" highestrms2)
(fmt-pretty FMT-val nil "~a" highestrms)
(fmt-pretty FMT-val nil "~a" highestpeak)

(format nil "Hightest RMS is ~a (~a dB)~%Highest peak is ~a~% Dynamic range (crest factor) based on highest RMS is ~a" highestrms highestrms2 highestpeak crest-highestrms)

And the supplemental function to delete the SCRATCH variables between each scan:

;nyquist plug-in
;version 4
;type analyze
;name "Delete Highest RMS variables"
;action "Delete SCRATCH variables from plugin Highest RMS…"
;info "By Holger Burghardt under GPL v3"
;author "Holger Burghardt\nHBB Think Tank"
;release 1.0.0
;copyright "Released under terms of the GNU General Public License version 3"

;; Released under terms of the GNU General Public License version 3:
;; https://www.gnu.org/licenses/gpl-3.0.html
;;
;; For information about writing and modifying Nyquist plug-ins:
;; https://wiki.audacityteam.org/wiki/Nyquist_Plug-ins_Reference

;; This plugin uses code from Audacity’s Nyquist forum:
;; https://forum.audacityteam.org/t/code-to-select-audio/53816/1 (primarily)
;; https://forum.audacityteam.org/t/acx-check-help-to-understand-its-code/53759/1 (to a much lesser extent)
;; Special thanks to Steve Daulton

(remprop '*SCRATCH* 'hbb-album-highestrms)
(remprop '*SCRATCH* 'hbb-album-highestpeak)

I know it still needs a few touches. For example, it’d be great if it would loop through all selected tracks and only then display (once) the final result. Second, it’d be equally great if I could calculate the RMS over all selected tracks. But I guess that would require all tracks to be loaded into the memory. Or can it be calculated with the number of samples in each track and the result of rms for each track?

steve · August 31, 2019, 6:28pm

Why are you using scratch? (I think I can guess why, but I’m not sure “precisely” what you are intending).

HBB.ThinkTank · August 31, 2019, 7:59pm

As far as I can tell, the plugin is started again (the tutorial speaks of “one invocation of a plugin”) on each track. So the values of the variables can not be taken from one track to the next. If I want to find the highest RMS over all tracks (say, of an album) without changing the relative loudness between the tracks, I need to store them in a way that survives each invocation. If I select only one track, the code would work the same way as if it hadn’t a scratch. It’s similar to ReplayGain which has a value for each track and a joint value for the whole album.

Am I missing something?

steve · August 31, 2019, 8:07pm

That’s what I thought you are doing.
Since that’s the case, it is possible to include the “clean up” code in the main plug-in, which will be more convenient than having to run it separately.
I need to do something else right now, but I’ll get back to you with some example code.

HBB.ThinkTank · August 31, 2019, 11:03pm

I’ve found on old thread from 2012 about Information passed from Audacity to Nyquist. Has anything come from it?

This problem (about manually deleting the scratch data) would have been solved with the proposal to pass on the info about the first selected track. Similarily my problem with a display after every track instead of only once at the end would habe been solved with an info about the number of selected tracks (it would also serve a solution to the scratch problem).

steve · September 1, 2019, 3:18pm

Yes. Audacity now makes a lot more information about the project available to Audacity. This is done through “global variables” and “property lists”. More information here: Missing features - Audacity Support

Yes.
You can get a list if which tracks are selected from the TRACKS property of the symbol SELECTION.
I see that you have already used property lists with the SCRATCH symbol. Documentation is here:
https://www.audacity-forum.de/download/edgar/nyquist/nyquist-doc/xlisp/xlisp-man/xlisp-man-014.htm

HBB.ThinkTank · September 1, 2019, 3:42pm

Thanks!

I saw those variables but (to my own shame) never really looked closely at them. So I’ll probably use ‘selection’ tracks both clean up scratch and display the final results only.

Question: Looking at the code on that page it doesn’t seem to matter whether the global variables are spellt in capitals or not (SCRATCH vs scratch)? Though I probably keep spelling them in capitals, just to visually identify them as such.

Btw, I am trying to add the (full) RMS (instead of only the highest) for the selection. Does rms necessarily require a let? I tried to no avail…

;version 4

(if (arrayp *track*)
  (progn
    (setf fullrms-l (* (rms (aref *track* 0))(rms (aref *track* 0))))
    (setf fullrms-r (* (rms (aref *track* 0))(rms (aref *track* 1))))
    (sqrt (/ (+ fullrms-l fullrms-r) 2.0)))
  (rms *track*))

steve · September 1, 2019, 4:08pm

Nyquist is traditionally written in lower case, though it is (mostly) case insensitive.
A notable exception to this is “aud-do” commands (Nyquist-Macros - Audacity Manual)
AUD-DO sends an Audacity “scripting” command as a string. The command in the string is NOT a Nyquist command, it’s an Audacity scripting command (a “Macro” command). Any “enum” values in a Macro command are case sensitive.

Most “special” variables (such as track, float-format, scratch …) have a star at the start and end. This is a convention that helps to visually identify them as such.

No, it doesn’t require a LET.

There’s a couple of problems with your code:

(setf fullrms-l (* (rms (aref *track* 0))(rms (aref *track* 0))))

This will evaluate (rms (aref track 0)) twice, which is needlessly inefficient.
If you wrote:

(setf rms-left (rms (aref *track* 0)))

Then you can use rms-left multiple times without having to re-analyze the track each time.
or you could do the same but with a local binding (so that rms-left is a local variable within the scope of a block, rather than a global variable)

(let ((rms-left (rms (aref *track* 0))))
  ... code within the LET block ...)

The multiplication symbol “*” works with numbers but not with sounds. You need to use MULT for sounds.
Similarly, SQRT works with numbers but not sounds. You need to use S-SQRT or SND-SQRT for sounds.

steve · September 1, 2019, 5:14pm

Your “highest peak” isn’t actually the highest peak.

Your “(setf mypeak” code sets “mypeak” to the average of the peak level of the left channel and the peak level of the right channel. If one channel is silent, then “mypeak” will only be half the actual peak value.

HBB.ThinkTank · September 1, 2019, 7:52pm

Right, my mistake, it should have been (max peak-l peak-r)!

RMS - I’ll look into it. I tried to adapt your code from rms.ny (your Measure RMS plugin)

steve · September 1, 2019, 9:38pm

Spoiler alert - If you want to work it out yourself, don’t scroll down the code below.

Here:

;nyquist plug-in
;version 4
;type analyze
;name "Measure Highest RMS"
;action "Measuring RMS level of the loudest section"
;release 1.0.1
;copyright "Released under terms of the GNU General Public License version 2"

;; Released under terms of the GNU General Public License version 2:
;; http://www.gnu.org/licenses/old-licenses/gpl-2.0.html


(defun number-to-string (number digits)
  ;; Returns the string representation of
  ;; a number with specified decimal places.
  (setf *float-format* (format nil "%.~af" digits))
  (format nil "~a" number))

(defun peak2 (sig &key (maxlen ny:all) (high 0))
  ;; A multi-channel version of PEAK.
  ;; Returns the maximum of "high" (keyword) and peak per channel.
  (if (arrayp sig)
      (dotimes (i (length sig) high)
        (setf high (max high (peak (aref sig i) maxlen))))
      (peak sig maxlen)))

(defun mean-sq (sig &optional (rate 100.0) window-size)
  (let (rslt step-size)
    (when (not (soundp sig))
      (error "Type  error in mean-sq" sig))
    (setf step-size (round (/ (snd-srate sig) rate)))
    (when (null window-size)
      (setf window-size step-size))
    (snd-avg (mult sig sig) window-size step-size OP-AVERAGE)))

(defun rms2 (sig &optional (rate 100) window-size)
  ;; Like RMS but also supports stereo
  (cond
    ((soundp sig)
      (if window-size
          (rms sig rate window-size)
          (rms sig rate)))
    ((arrayp sig)
      (let ((ms-left (mean-sq (aref sig 0)))
            (ms-right (mean-sq (aref sig 1))))
        (setf mean (mult 0.5 (sum ms-left ms-right)))
        (s-sqrt mean)))
    (t (error "Type error in rms2" sig))))

(defun peak-rms (sig rate val)
  ;Return the max of 'val' and the peak rms of 'sig'
  (let ((peak-rms (peak (rms2 sig rate) ny:all)))
    (max val peak-rms)))

(defun process-first-track (sig rate)
  (putprop '*scratch* (peak2 sig) 'peak)
  (putprop '*scratch* (peak-rms sig rate 0) 'rms))

(defun process-others (sig rate)
  (let ((old-peak (get '*scratch* 'peak))
        (old-rms (get '*scratch* 'rms)))
    (putprop '*scratch* (peak2 sig :high old-peak) 'peak)
    (putprop '*scratch* (peak-rms sig rate old-rms) 'rms)))

(defun output ()
  (let ((peak (get '*scratch* 'peak))
        (peak-db (linear-to-db (get '*scratch* 'peak)))
        (rms (get '*scratch* 'rms))
        (rms-db (linear-to-db (get '*scratch* 'rms))))
    (remprop '*scratch* 'peak)
    (remprop '*scratch* 'rms)
    (format nil
            "Hightest RMS is ~a (~a dB).~%~
            Highest peak is ~a (~a dB).~%~
            Crest factor based on highest values: ~a (~a dB)."
            (number-to-string rms 3)
            (number-to-string rms-db 1)
            (number-to-string peak 3)
            (number-to-string peak-db 1)
            (number-to-string (/ peak rms) 2)
            (number-to-string (linear-to-db (/ peak rms)) 1))))


(setf rms-rate 10.0)

(let ((selected (get '*selection* 'tracks))
      (id (get '*track* 'index)))
  (if (= id 1)
      (process-first-track *track* rms-rate)
      (process-others *track* rms-rate))
  (if (= id (length selected))
      (output)
      ""))

max-rms.ny (3.03 KB)

HBB.ThinkTank · September 2, 2019, 5:51am

Well, well, well, Steve, you’ve had fun with the code.

I need to carefully investigate your code.

I already like two ideas. I would have used (when…) for the output, which so far didn’t work. Besides, your solution is more elegant. And there’s the function number-to-string…

The remaining code needs deeper inspection

I would add an ordinary RMS over all selected tracks to complete the info on an selection (album).

And a control to allow users to chose the length of the time window.

steve · September 2, 2019, 8:15am

I predicted that you might want to do that , so it’s dead easy. Just remove or comment out the line:

(setf rms-rate 10.0)

Then either replace it directly with a control, for example:

;control rms-rate "RMS rate" float "Hz" 10 1 100

or replace with a “widow size” control (the inverse of the rate), for example:

;control rms-window "RMS window size" int "ms" 100 1 1000

(setf rms-rate (/ 1000.0 rms-window))

(I used an integer slider above, because I think it looks nicer, and I doubt that fractions of a millisecond are going to be useful for anyone.)

HBB.ThinkTank · September 2, 2019, 9:21pm

There’s somewhere a change in the functions.

I have tested it on a selection of three tracks,* and my own code results in…

Highest RMS is 0.362093 (-8.82359 dB)
Highest peak is 0.977234
Dynamic range (crest factor) based on highest RMS is 9 with 7039536 samples

…while your code returns:

Highest RMS is 0.563 (-5.0 dB).
Highest peak is 0.977 (-0.2 dB).
Crest factor based on highest values: 1.74 (4.8 dB).

Since I’m still trying to get my head around your code (way beyond my beginner’s level ) I can’t say right now what the problem might be.

Let’ try and break down your code:

number-to-string function

This one’s easy enough to understand, as we’ve already talked about it here. A function to format a number with two arguments:

The number to format
A number of digits indicating the number of post-comma decimals

peaks2 function

This one already trickier, as it has some features I’m not yet familiar with. sig seems to be a placeholder variable for whatever sound you want to feed to it, and it is my understanding that chosing the string sig is rather a personal choice than obligatory. In our plugin it eventually corresponds to track.

But the other argument means nothing to me: &key (maxlen ny:all) - well actually, I already came across maxlen for peak, it limits the evaluation to maxlen=x samples. (high 0) I believe resets the variable to 0 at each calling of the function.
(if (arrayp sig) does check wheter the track is stereo or mono or indeed a track with x channels…
dotimes creates a loop, once for each channel as (length sig) will return the number x of channels, because sig = track will be an array of x mono tracks.

The rest is “simple” peak expression voodoo
I just don’t understand why there’s a high as a result expression in (dotimes (i (length sig) high).

mean-sq function

I must admit that I am perplexed by this function. It contains some error management (error…) and the code to calculate RMS. Why have you chosen the operator snd-avg over rms? My guess is that this operator allows a better control over the window size but I must admit I haven’t understood snd-avg and the blocksize and stepsize. I mean, the concept: yes, but not what it actually does.

rms2 function

(cond…) does (to me) the same as (if (arrayp sig) except allowing for a third option in case someone feeds something other than sound to this function.
I don’t get why rms has rate and window-size as optional arguments when “window size is 1/rate seconds” (I know I’m missing something, probably again the understanding of window-size and its relation to rate).

Doesn’t this function simply calculate the RMS?

peak-rms function
“Simply” calculates the RMS across multiple tracks

The remaining code seems to me straight-forward enough! (Did I really say that?)

The Alan Parsons Project - Tales Of Mystery And Imagination, early 2000s pressing of the 1987 remix: tracks “The Fall of the House of Usher (instrumental): III Intermezzo”, “07. The Fall of the House of Usher (instrumental): II Arrival” and “10. The Fall of the House of Usher (instrumental): V Fall”

HBB.ThinkTank · September 2, 2019, 10:09pm

And the peak2 function can be rewritten:

(defun peak2 (sig &key (maxlen ny:all) (high 0))
  ;; A multi-channel version of PEAK.
  ;; Returns the maximum of "high" (keyword) and peak per channel.
  (if (arrayp sig)
      (dotimes (i (length sig) high)
        (setf high (max high (peak (aref sig i) maxlen))))
      (peak sig maxlen)))

to

(defun peak2 (sig &key (maxlen ny:all) (high 0))
  ;; A multi-channel version of PEAK.
  ;; Returns the maximum of "high" (keyword) and peak per channel.
  (cond
    ((soundp sig)
	  (if maxlen
	    (peak sig maxlen)
		(peak sig)))
	((arrayp sig)
      (dotimes (i (length sig) high)
        (setf high (max high (peak (aref sig i) maxlen)))))
    (t (error "Type error in peak2" sig))))

Right?

Only two questions remain: what does ny:all stand for and why (dotimes (i (length sig) high)?