Macro/mod-script strange behavior

Hi there.
I need to process lots of voice recordings tracks, but i need to spare the ‘Intro’ in the tracks which is seated at the start and separated by a 2-3 seconds silence from the rest. I ‘wrote’ a script, inspired from the ‘SilenceMarker’ plugin (thanks Steve) just to get the time after the intro. Then i call the AUD-Do "Select: function to feed the start and end of the track to the batch processing with Python in mod-script-pipe.

When i feed the script to the Nyquist prompt (using legacy) the script works well and select nicely what i need .
But the trouble comes when i used the script as a plugin in Python. Audacity works 2 times of 10 hanging dry.
I notice that for the plugin to be ‘allowed to work’ the track must be selected but I think the (aud-do “SelectAll” command that i inserted script has something to do with it. The reason is when launching the macro alone (from the macro window ) the good selection is done but it reverts instantaneously to select the whole track as the command finished (is really blinking). For now i have a convoluted solution to make it work using the macros way but i would like to use Python and of course know what i’m doing wrong.

Env: Linux, Audacity 2.3.3 compiled with mod-script-pipe, Python 3.

Thanks in advance

The first thing I’d suggest is that we update that script so use current version syntax. If you attach the script to your reply I’ll show you how to do that.

Yes. Nyquist plug-ins can only access audio from tracks when that audio is selected.

I’m not sure what you mean. Do you mean that you inserted (aud-do “SelectAll”) into the Nyquist script or into your Python script? (both would be wrong, but for different reasons).

The reason it won’t work correctly in the Nyquist script:
Before a Nyquist script runs, Audacity passes certain information about the current state of the Audacity project to Nyquist, along with the Nyquist code. The Nyquist code then runs, and may use some, all, or none of that information. That “information” may include a pointer to the selected track audio. The important part is that this “information” is a one time exchange, and is the only information that Nyquist can get from Audacity. The information can’t be updated while Nyquist is running.

Example:
Say you have a track selected to the first 10 seconds of a track.
Nyquist receives a pointer to the first 10 seconds of audio (in modern syntax, that pointer is the symbol track)
If you then use an “aud-do” command to select the second 10 seconds, then the selection will change, but track still points to the first 10 seconds. There is currently no way to access the updated selection.


The reason it won’t work correctly in Python:
“(aud-do …” is a Nyquist command, not a Python command.


The solution is to use Python scripting commands to select the required audio, and then call the Nyquist command.

Unfortunately there is a bug in the scripting API: Calling the Nyquist Prompt from Python does not work properly. To work around this bug, the Nyquist script needs to be made into a plug-in and installed in Audacity. Python will then be able to call the plug-in and run the script correctly.

Well Thanks a lot Steve for the information and the detailed explanation. I’ve suspected something of this ‘syncro’ nyquist-Audacity thing, because this go-no-go results. And yes i knew (audo-do is a wrap for some base nyquist functions.

As update i discover a working solution for my problem when I saw that launching from Python an AUD-DO SELECT command with random harcodedValues, the selection was done perfectly. So I looked for a way to recover the data from my script and sent them back to Python to do the same but i didn’t find much info to do this.

I must say that the solution in fact is a dirty hack. Perhaps there is a ‘proper’ way to do it. Also I 'd like to share this method for other people who need audacity to return data from any script you launch in Audacity (of course beware the evil scripts) as it saves me a lot of work.

My solution: In the script, once i the function find the values for the selection: Start and end of the track (here no problems) i inserted a “write” command to the “pipe file” (in Linux is located in “tmp/audacity_script_pipe.from.1000”) and with my values inside a string.

(setq fp (open "/tmp/audacity_script_pipe.to.1000" :direction :output)) 
(Setq res (format  nil "End=~S Start=~S:" endT StartT)) ; the returned string  
(print res fp)

(One Precision, the string formed must be finished with a colon ( to be interpreted as a ‘failed’ command).

The Python client waiting for the answer recovers a ‘fail command’ but with my string attached. (that was a nice surprise) Then just Python get the data and put them into a working AUD-DO SELECT command this time with my values.

Till now more than 80 mp3 files have been processed and no hiccups.

My script:
AuCompressIntro.ny (2.56 KB)

This is an updated version of SilenceMarker.ny

;nyquist plug-in
;version 4
;type analyze
;name "Silence Finder"
;manpage "Silence_Finder"
;author "Steve Daulton"
;release 2.4.0
;copyright "Released under terms of the GNU General Public License version 2"

;; Original version by Alex S. Brown, PMP (http://www.alexsbrown.com)
;;
;; Released under terms of the GNU General Public License version 2:
;; http://www.gnu.org/licenses/old-licenses/gpl-2.0.html
;;
;; For information about writing and modifying Nyquist plug-ins:
;; https://wiki.audacityteam.org/wiki/Nyquist_Plug-ins_Reference

;control threshold "Treat audio below this level as silence (dB)" float "" -30 -100 0
;control min-silence "Minimum duration of silence (seconds)" float "" 1.0 0.1 5.0
;control label-position "Label placement (seconds before silence ends)" float "" 0.3 0.0 1.0

(setf threshold (db-to-linear threshold))
; Label is usually offset to earlier time.
(setf label-position (- label-position))

(setf *labeltxt* "S")
(setf *labels* NIL)


(defun to-mono (sig)
  ;;; coerce sig to mono.
  (if (arrayp sig)
    (s-max (s-abs (aref sig 0))
           (s-abs (aref sig 1)))
    sig))

(defun reduce-srate (sig)
  ;;; Reduce sample rate to (about) 100 Hz.
  (let ((ratio (round (/ *sound-srate* 100))))
    (snd-avg sig ratio ratio OP-PEAK)))

(defun add-label (samples srate offset)
  ;;; Add new label to *labels*
  (let ((time (+ (/ samples srate) offset)))
    (push (list time *labeltxt*) *labels*)))

(defun format-time (s)
  ;;; format time in seconds as h m s.
  (let* ((hh (truncate (/ s 3600)))
         (mm (truncate (/ s 60))))
  ;i18n-hint: hours minutes and seconds. Do not translate "~a".
  (format nil "~ah ~am ~as"
      hh (- mm (* hh 60)) (rem (truncate s) 60))))

(defun label-silences (sig)
  ;;; Label silences that are longer than 'min-len' samples.
  (let* ((sample-count 0)
         (sil-count 0)
         (srate (snd-srate sig))
         (min-len (* min-silence srate)))
    (do ((val (snd-fetch sig) (snd-fetch sig)))
        ((not val) sil-count)
      (cond
        ((< val threshold)
            (incf sil-count))
        (t  (when (> sil-count min-len)
              (add-label sample-count srate label-position))
            (setf sil-count 0)))
      (incf sample-count))
    ;; If long trailing silence, add final label at 'min-silence' AFTER last sound.
    (when (> sil-count min-len)
      (setf final-silence (- sample-count sil-count))
      (add-label final-silence srate min-silence))
    *labels*))


;;  Bug 2352: Throw error if selection too long for Nyquist.
(let* ((dur (- (get '*selection* 'end)
               (get '*selection* 'start)))
       (samples (* dur *sound-srate*))
       (max-samples (1- (power 2 31))))
  (if (>= samples max-samples)
      (format nil "Error.~%Selection must be less than ~a."
              (format-time (/ max-samples *sound-srate*)))
      ;; Selection OK, so run the analyzer.
      (let ((sig (reduce-srate (to-mono *track*))))
        (setf *track* nil)  ;free *track* from memory
        (if (label-silences sig)
            *labels*
            "No silences found.
Try reducing the silence level and
the minimum silence duration."))))

Could you describe what the objective is of your entire Python / Nyquist scripting task.

To return a string to Python from Nyquist, you could do something like this:

;type tool
(setf var 42)

;; See https://www.audacity-forum.de/download/edgar/nyquist/nyquist-doc/xlisp/xlisp-ref/xlisp-ref-121.htm
(let ((fp (open "/tmp/audacity_script_pipe.from.1000" :direction :output)))
  (format fp "Hello World.~%The anwser is ~a~%" var)
  (close fp))

"" ;Return an empty string as a valid no-op

This is slightly abusing the system, but we’re not relying on failed commands.
“fp” (file pointer) points to the output stream, which is the “from” pipe.
I make no guarantees whether this will work on platforms other than Linux.

It should also be noted that this assumes that the UID (user identifier) is 1000 (which it usually is on a single user system). A more general solution (for Linux) would be to send the UID from Python to the Nyquist script as a ;control parameter.

Thanks for finding the ‘legal’ way to get the server response and a nice point about passing the UID parameter from client to server.

To give some background in this thread, I’m working for an Association without lucrative purpose, making audio books for blind, impaired vision people or people with some other kind of disabilities.

The books are divided in tracks with lengths that are easy to manage for the final user. This parts will be the tracks to burn into CD’s.

We have several readers who get the book with the list of ‘tracks’ . They record the audio into several files (to make the reading process less tiresome and to stop the reading when they make an error) . These files are then sent for editing

The editor (i am also one) check, clean, cut and append the different files in one track AND insert at the beginning of the track a ‘Track identification’ which is a short audio (with music) describing the track number.
Then all the ‘book’ edited pass a final check and normalization of the audio. The main software used is Soundforge and also some use audacity.

We are all volunteers and mostly with a basic or sometimes none knowledge in computing. I am one exception as I have been programming for a long time (nowadays mostly in C and Python). I had a master in AI and it was there where a i ‘learned’ Lisp, (long time ago) so for all this I get promoted to final-corrector (lucky me).

So, between other checks, I do the filtering of the tracks but avoiding to touch the ‘track identification’ piece, otherwise this part gets ‘ugly’ when applying compression.
(Dynamic compression filter from Chris Capel et al., best filter out there for speech, thanks everyday. )
It’s absolutely necessary to do that? … the devil is in the details, and for the people listening i think we can go the extra mile.

About the ‘technical part’, this ‘intro’ piece contains an audio of variable duration (generally below 7seconds) padded with 2 seconds silence at the start and aprox 2 seconds silence at the end (depends on the editing job).
To bypass this first part you must select the track from the point between the end of the ‘intro’ and the start of the actual reading. Manually is easy but time consuming (and boring) when you get lots of files.
Other solutions considered: Do the the filtering before inserting the ‘intro’, but it will do more harm than good because the editors aren’t prepared to do it. Doing it after, will be messing with the editors well established process and me scripting anyway.

So at first I’ve been doing this with the help of python, ffmpeg and AU. Ffmpeg to cut the desired part then get AU to apply the filter chain and finally concatenated all with ffmpeg. Cumbersome but It worked well.
Recently I invested a little more time to reunite some scripts I’ve written to do other tasks for the job (tag’in the files, volume optimizing etc) inside a Python gui so I decided to learn the nyquist/macro way to optimize the filtering part, and that’s why i landed here.

Now the mod-scripting and the macro ways work but (sorry to say) in their convoluted way.

The End (for now…)

Sorry to get so long (you asked)

So am I right in thinking that the reason you want to get data back from Audacity into Python, is so that you can select from the end of the first silence to the end of the track?

Sorry for the delay to answer. Really complicated life now.
Yes. In fact from the ‘second’ full silence to bypass the musical intro.

I’ve re-read your description of the job, and if I understand correctly, each segment of audio is a separate audio clip. If that is the case, then you can get the start and end times of each audio clip with the command:

"GetInfo: Type=Clips Format=JSON"

Thanks for pointing that but in my case there aren’t clips. I received the tracks in already edited files, the only way to find the starting point of the audio to process is finding the silent gap which follows the musical intro.

So how about using “Silence Finder” (Silence Finder - Audacity Manual) and then

"GetInfo: Type=Labels Format=JSON"