Different Errors running pipe test

[Hoperin]

Win 11 / audacity 3.1.3 / python 3

I’m trying to get a python script running that processes a bunch of files in a bunch of different folders in mp3s.
I haven’t done any work previously with macros or pipes, but I’m been making fair progress.
My problem is that sometimes when I run pipe-test.py, I get one of 2 errors, and sometimes it gets through and runs the rest of my script (which doesnt quite work but first I’m trying to get the test to work consistently.

My best chance at getting it to run throught the test and move on is to run it off a fresh restart, otherwise sometimes if i restart audacity it will work, but most times i get one of these errors, and sometimes a restart doesnt give a success either:

"C:\Program Files\Python310\python.exe" C:/Users/herin/Documents/GitHub/audacity/scripts/piped-work/pipe_test.py
pipe-test.py, running on windows
Write to  "\\.\pipe\ToSrvPipe"
Read from "\\.\pipe\FromSrvPipe"
-- Both pipes exist.  Good.
Traceback (most recent call last):
  File "C:\Users\herin\Documents\GitHub\audacity\scripts\piped-work\pipe_test.py", line 46, in <module>
    TOFILE = open(TONAME, 'w')
FileNotFoundError: [Errno 2] No such file or directory: '\\\\.\\pipe\\ToSrvPipe'

Process finished with exit code 1

or this one (while audacity is still definitely running, and pipe is definitely enabled)

"C:\Program Files\Python310\python.exe" C:/Users/herin/Documents/GitHub/audacity/scripts/piped-work/pipe_test.py
pipe-test.py, running on windows
Write to  "\\.\pipe\ToSrvPipe"
 ..does not exist.  Ensure Audacity is running with mod-script-pipe.

Process finished with exit code 0

I’m not changing anything in the script, just closing and reopening audacity trying to get a success.
I’ve clones the entire audacity github repo and am working straight out of the pipe-test.py, the only changes I’ve made are adding sleep:

print("-- Both pipes exist.  Good.")
sleep(0.01)
TOFILE = open(TONAME, 'w')

as well as my code following the pipe check that does sometimes get reached :

baseDirectory = "C:\\Users\\herin\\Desktop\\PollyGVH\\"

files = GetAllFiles()

for file in files:
    mp3Dir = os.path.splitext(file)[0]
    wavDir = mp3Dir + "_wav"
    if not os.path.exists(mp3Dir):
        os.makedirs(mp3Dir)
    if not os.path.exists(wavDir):
        os.makedirs(wavDir)

    mp3s = GetAllMP3s()

    for mp3 in mp3s:
        #do_command(f"Open: Filename={mp3}")  # should be .mp3
        #do_command(f"ExportWav: Filename = {mp3}")

Any idea what could be causing such unpredictability?

[steve]

Write to  "\\.\pipe\ToSrvPipe"
Read from "\\.\pipe\FromSrvPipe"
-- Both pipes exist.  Good.
Traceback (most recent call last):
  File "C:\Users\herin\Documents\GitHub\audacity\scripts\piped-work\pipe_test.py", line 46, in <module>
    TOFILE = open(TONAME, 'w')
FileNotFoundError: [Errno 2] No such file or directory: '\\\\.\\pipe\\ToSrvPipe'

This part is interesting because “print(”-- Both pipes exist. Good.“)” comes after the script has checked that “ToSrvPipe” exists, but then when Python tries to write to it, it’s apparently not there any longer.

Unfortunately it is not so easy to monitor named pipes in Windows as other platforms, but according to this post you can see them with “Process Explorer”: https://stackoverflow.com/questions/258701/how-can-i-get-a-list-of-all-open-named-pipes-in-windows
It could be worth playing with that to see if/when the pipes are appearing/disappearing.

Also, please be aware that pipe-test.py is not intended for practical use, but rather as a quick test to check that mod-script-pipe is working.

[JohnMayo]

I’m having problems using the named pipe interface from Python to Audacity. The named pipes don’t seem to always exist. I’m running with mod-script-pipe enabled. Sometimes it works fine. Other times I can’t get it to work at all.

Looking at the list of named pipes on my machine when I have Audacity running seems to show the \.\pipe\ToSrvPipe and \.\pipe\FromSrvPipe to exist right up until I try to use then and then they vanish. I’m not sure what is killing the named pipes but it almost seems like it is the attempt to use them that might be doing it.

I’ve had times when repeated attempts to use the pipes fails a few times and then magically works. I’m really not sure what is going on.

I’m on Windows 11 and I’m using the code from pipe-test.py.

If the pipe-test.py isn’t intended for practical use then is there some code that is?

I’d really like to automate some of my Audacity workflow using Python but it is hard to do that when I can’t reliably connect to Audacity from Python.

[JohnMayo]

A little more insight into what I’m doing and seeing…

I launch Audacity 3.4.1 on Windows 11 and confirm the FromSrvPipe and ToSrvPipe both exists in file://.//pipe// in my web browser. I run pipe_test.py and get the following output:

pipe-test.py, running on windows
Write to “\.\pipe\ToSrvPipe”
Read from “\.\pipe\FromSrvPipe”
– Both pipes exist. Good.
Traceback (most recent call last):
File “C:__JKM\Syncthing\DevProjects\末NoRepo\Audacity\pipe_test.py”, line 44, in
TOFILE = open(TONAME, ‘w’)
^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: ‘\\.\pipe\ToSrvPipe’

At this point, sometimes the FromSrvPipe and ToSrvPipe both still exists and running pipe_test.py magically works. Other times, FromSrvPipe and ToSrvPipe no longer exist and the output is:

pipe-test.py, running on windows
Write to “\.\pipe\ToSrvPipe”
…does not exist. Ensure Audacity is running with mod-script-pipe.

While this failure makes sense given the pipes are closed, it means the process is unreliable.

[steve]

Just a thought:

Try running the Python script from a normal directory rather than a subdirectory of Syncthing. (I’m assuming that the Syncthing directory is a dynamically synchronising folder (https://syncthing.net/) and that may be interfering with the Python script’s operation.

[JohnMayo]

I’ve moved the code outside of SyncThing. While I doubt it would cause the named pipes created by Audacity to close, it was worth a shot. After moving the Python code outside of SyncThing, I’m seeing the same behavior.

Sometimes when I close Audacity after a failed attempt I get a crash report indicating:

Crash reason: 0x40000015 / 0x00000001
Crash address: 0x7ffcd339baf6
Process uptime: 138 seconds

The problem seems like it might be on the Audacity side of things.

[JohnMayo]

I’ve put the code that makes the named pipe connection into a loop so it gets retried a few times if it fails. Sometimes it fails a few times and then is magically able to make the connection fine. Same code, no changes in Audacity or with the named piped that I’m aware of.

[JohnMayo]

One other thing I’ve been seeing is problems with writing the config file:

I’m not sure if it is related or not but I’ve seen it enough times that I thought it was worth mentioning.

The problem with the config file seemed to revert the mod-script-pipe setting back to “new”.

And I’ve also just gotten this error:

[steve]

Did it work in your previous version of Audacity? If it did, then it may be worth going back to that version.
Let us know if it works in an old version, as that would indicate a bug in the current version that needs to be logged and fixed.

[JohnMayo]

I had some stuff working in a prior version but only did some token testing. When I hit some problems getting the named pipes working I made sure to get on the latest version. What version of Audacity do you recommend?

…

I’m seeing similar behavior in Audacity 3.3.3.
…

I’m seeing similar behavior in Audacity 3.1.2.
…

I’m seeing similar behavior in Audacity 3.0.2.

I’m going to switch back to Audacity 3.4.1.

…

Still playing around with this. Still having problems and seeing Audacity crashing:

I’ve been using Audacity for around 15 years or so and I haven’t seen it crash like this in ages. No doubt that is simply a reflection of the automation code being knew and not getting as much use as the core functionality of the codebase. I’m still not seeing any sort of clear pattern in what I’m doing that might shed some light on the root cause of the problems.

[steve]

@JohnMayo

When I hit some problems getting the named pipes working

Was that in the older version?
If scripting was working in the older version, and then it stopped working in the older version, that suggests that the problem lies elsewhere on your computer.

There are several bugs relating to macros that have been reported in Audacity 3.4.0 / 3.4.1. Some of them have now been fixed for the next Audacity release.

@JohnMayo

I’m seeing similar behavior in Audacity 3.3.3

I’m not aware of any major bugs in Audacity 3.3.3’s macros, so that might be a good version to go back to while you troubleshoot the problem. See if you can identify reliable steps in 3.3.3 that trigger the problem.

[JohnMayo]

I’ve switched back to 3.3.3. (Had I known that version 3.4.x had known problems with macros, I wouldn’t have switched back to that version yesterday.)

Running Audacity 3.3.3 immediately after installing it generates a bunch of 'Unable to load the “mod-___” module for a variety of modules. Presumably this is due to having installed 3.3.3 on top of the 3.4.1 install and parts of 3.4.1 not getting properly cleaned out. I’ll completely remove Audacity and reinstall it again to see if that still happens…

In the process of completely uninstalling Audacity I noticed that I had Audacity folders in both C:\Program Files (x86) and C:\Program Files. I’m not sure how/when that happened. Presumably it happened when I installed one of the older versions yesterday. I did default installations so presumably one of the older versions installed into C:\Program Files (x86).

I’ve removed all traces of Audacity I could find and did a clean, default installation of Audacity 3.3.3 and enabled the mod-script-pipe module and restarted Audacity., confirmed the ToSrvPipe and FromSrvPipe exist. (Those are really unclean names for the named pipes. Any chance that a parallel set that are clearly named as being to/from Audacity could be created at some point? I get the need to leave the existing pipe names in place so older code doesn’t get broken. But having two sets that are work in parallel seem like it should work.)

Running pipe_test.py:

pipe-test.py, running on windows
Write to “\.\pipe\ToSrvPipe”
Read from “\.\pipe\FromSrvPipe”
– Both pipes exist. Good.
Traceback (most recent call last):
File “C:\DevProjects\Audacity\pipe_test.py”, line 44, in
TOFILE = open(TONAME, ‘w’)
^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: ‘\\.\pipe\ToSrvPipe’

Running pipe_test.py a second time got the same results.

Running pipe_test.py a third time got these results:

pipe-test.py, running on windows
Write to “\.\pipe\ToSrvPipe”
Read from “\.\pipe\FromSrvPipe”
– Both pipes exist. Good.
– File to write to has been opened
– File to read from has now been opened too

Send: >>>
Help: Command=Help
Rcvd: <<<

{ “id”:“Help”, “name”:“Help”, “params”:
[
{ “key”:“Command”, “type”:“string”, “default”:“Help” },
{ “key”:“Format”, “type”:“enum”, “default”:“JSON”, “enum”:
[ “JSON”, “LISP”, “Brief” ] } ],
“url”:“Extra_Menu:_Scriptables_II#help”,
“tip”:“Gives help on a command.” }
BatchCommand finished: OK

Send: >>>
Help: Command=“GetInfo”
Rcvd: <<<

{ “id”:“GetInfo”, “name”:“Get Info”, “params”:
[
{ “key”:“Type”, “type”:“enum”, “default”:“Commands”, “enum”:
[ “Commands”, “Menus”, “Preferences”, “Tracks”, “Clips”, “Envelopes”, “Labels”, “Boxes” ] },
{ “key”:“Format”, “type”:“enum”, “default”:“JSON”, “enum”:
[ “JSON”, “LISP”, “Brief” ] } ],
“url”:“Extra_Menu:_Scriptables_II#get_info”,
“tip”:“Gets information in JSON format.” }
BatchCommand finished: OK

Audacity was untouched between runs of pipe_test.py.

Does Audacity need a little time after starting before it can accept commands? If so, how long does it need?

I continued to run pipe_test.py and it worked fine until the 15th run at which point it was back at the initial error results:

pipe-test.py, running on windows
Write to “\.\pipe\ToSrvPipe”
Read from “\.\pipe\FromSrvPipe”
– Both pipes exist. Good.
Traceback (most recent call last):
File “C:\DevProjects\Audacity\pipe_test.py”, line 44, in
TOFILE = open(TONAME, ‘w’)
^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: ‘\\.\pipe\ToSrvPipe’

Since I only used pipe_test.py for these tests, I think I can rule out my code as the problem.

Since I only used a clean, default install of Audacity 3.3.3 (on Windows 11), I think we can rule out the problem being caused by left over parts of the Audacity 3.4.1 install.

The problem seems to be that the named pipes are a bit unstable in Audacity 3.3.3 for some reason.

The only macro commands I’ve run are the Command=Help and Command=“GetInfo” from pipe_test.py.

If scripting was working in the older version, and then it stopped working in the older version, that suggests that the problem lies elsewhere on your computer.

The code I’ve been writing is new code and not something I’ve run with older versions of Audacity. I started on either version 3.3.3 or 3.4.0 and upgraded to 3.4.1 when I initially hit problems with the named pipes.

Steve, have you tried taking the code in pipe_test.py and putting it into a loop to see if it reliable worked for you?

[steve]

@JohnMayo

I noticed that I had Audacity folders in both C:\Program Files (x86) and C:\Program Files.

That’s because Audacity on Windows used to be a 32-bit app, and later became a 64-bit app.

[JohnMayo]

That’s because Audacity on Windows used to be a 32-bit app, and later became a 64-bit app.

I figured as much. Regardless, that doesn’t help me get the named pipes working.

I’ve written a short script to test the named pipes:

import os
import time

PIPE_TO_AUDACITY = '\\\\.\\pipe\\ToSrvPipe'
PIPE_FROM_AUDACITY = '\\\\.\\pipe\\FromSrvPipe'

def pipe_exists(pipe_path):
    return os.path.exists(pipe_path)


def check_pipes():
    print(f'Loop\tTo\tFrom')
    for loop in range(0,500): 
        if pipe_exists(PIPE_TO_AUDACITY):
             to_status = 'open'
        else:
             to_status = 'CLOSED'
        if pipe_exists(PIPE_FROM_AUDACITY):
             from_status = 'open'
        else:
             from_status = 'CLOSED'
        print(f'{loop}\t{to_status}\t{from_status}')
        time.sleep(1)


check_pipes()

If I run it and then open up Audacity, the pipes open up for a few second and then close and sometimes Audacity crashes when I close it.

Note that this test code doesn’t actually read or write from the name pipes. It just checks if they exist. It seems like the name pipes might be inherently unstable for some reason.

[steve]

@JohnMayo

I’ve removed all traces of Audacity I could find

Did you remove Audacity’s settings folder (where “Preferences” settings are)?
"Users<username>\AppData\Roaming\audacity"

[steve]

@JohnMayo

Any chance that a parallel set that are clearly named as being to/from Audacity could be created at some point?

Unlikely that those names will be changed any time soon. This scripting feature was added by the original development team. Audacity was acquired a couple of years ago by Muse Group, who seem to have different priorities.

[JohnMayo]

Did you remove Audacity’s settings folder (where “Preferences” settings are)?
“Users\AppData\Roaming\audacity”

Given I didn’t know about that folder, no, I did not remove it.

…

I’ve removed the folder (while Audacity was closed). I started my test script (see above for the code) and started Audacity. Since Audacity didn’t have the settings, I had to reset the mod_script_pipe to enabled, close and restart Audacity again. The pipes opened for about three seconds and then closed.

So it doesn’t look like removing the setting folder had any impact on the named pipe functionality.

Steve, are you able replicate the problem to any degree on your system? Or are the named pipes working perfectly fine for you?

[steve]

@JohnMayo

Does Audacity need a little time after starting before it can accept commands?

Probably. No applications start “instantly”. In complex apps there are often a lot of things that need to initialise. Nevertheless, Audacity should be able to receive commands pretty soon after the main Audacity window appears.

In my more recent version of “pipeclient.py” I allow 1/10th second for Python to connect to the pipes, but that assumes that Audacity is already up and running:

    def _write_thread_start(self) -> None:
        """Start _write_pipe thread"""
        # Pipe is opened in a new thread so that we don't
        # freeze if Audacity is not running.
        write_thread = threading.Thread(target=self._write_pipe_open)
        write_thread.daemon = True
        write_thread.start()
        # Allow a little time for connection to be made.
        time.sleep(0.1)
        if not self._write_pipe:
            sys.exit('PipeClientError: Write pipe cannot be opened.')

(Note that this version has not been widely tested as it was for my own use on Linux. The older “official” version ise here: https://github.com/audacity/audacity/blob/master/scripts/piped-work/pipeclient.py

[JohnMayo]

@steve

This scripting feature was added by the original development team. Audacity was acquired a couple of years ago by Muse Group, who seem to have different priorities.

What does that mean for fixes and/or enhancements to the scripting functionality? The documentation implies that scripting is supported functionality.

I’d really like to know if the project I’m working to try to automate my workflow in Audacity is worth continuing or not.

[JohnMayo]

@steve

No applications start “instantly”. In complex apps there are often a lot of things that need to initialise. Nevertheless, Audacity should be able to receive commands pretty soon after the main Audacity window appears.

In my more recent version of “pipeclient.py” I allow 1/10th second for Python to connect to the pipes, but that assumes that Audacity is already up and running:

If I start Audacity, wait for it to finished loading and then start my test script, the pipes seem to be open initially and then close after the first check:

Loop    To      From
0       open    open
1       CLOSED  CLOSED
2       CLOSED  CLOSED
3       CLOSED  CLOSED
4       CLOSED  CLOSED