Unattended command line processing

In another thread (https://forum.audacityteam.org/t/command-line-processing/15408/1) we talked about this and I started a new thread to present a “proof of concept”.

changes for unattended CLI operation
for unattended operation I think On Demand should (must?) be off

usage:
Audacity.exe -dounattended song.mp3

example:
create a “working” directory somewhere (here I am working in D:MP3sconverted)
open a CLI and cd to the “working” directory–this is the directory in which the file will be saved
run Audacity from the CLI with the -dounattended switch – only submit a single file to convert
D:MP3sconverted>“D:audioAudacityAudacity.exe” -dounattended D:MP3ssong.mp3

result:
D:MP3sconvertedsong.m4a is created with the user’s default M4A options

notes:
the new file gets the old file’s name with the extension changed to reflect the conversion
the complete path to Audacity must be specified as you do not want to be writing into your application directory
the fully qualified filename (complete with path and extension) must be used in the song name
without the -dounattended switch Audacity behaves as normal (except it will not nag to save an empty project)

to perform many of these conversions overnight create a script–either save it as a batch file and execute it or paste from a text editor into a CLI (I have tried pasting into a Win7 CommandLine and it does not work–using a batch file launches cmd.exe sequentially and works):

D:MP3sconverted>“D:audioAudacityAudacity.exe” -dounattended D:MP3ssong1.mp3
D:MP3sconverted>“D:audioAudacityAudacity.exe” -dounattended D:MP3ssong2.mp3
D:MP3sconverted>“D:audioAudacityAudacity.exe” -dounattended D:MP3ssong3.mp3

code changes are surrounded by pairs of comments with my initials:
//efm5
changed code
//efm5

below are all the changes with a few lines of context
line numbers below are approximations, correct to the SVN as of 20Aug2010
some of the reasoning is commented in the code,
some is described here

AudacityApp.h line #218:

 public:
    DECLARE_EVENT_TABLE()
//efm5
// store the current working directory as it exists when Audacity is launched
//(it gets hijacked somewhere -- see Project.cpp line #758
private:
   wxString mCLIpath;
   void SetCLIpath(const wxString & CLIpath) {mCLIpath = CLIpath;};
public:
   const wxString & GetCLIpath() const {return mCLIpath;};
//efm5
};

extern AudacityApp & wxGetApp();

AudacityApp.cpp line #873:

bool AudacityApp::OnInit()
{
//efm5
   SetCLIpath(wxGetCwd());
//efm5
#if defined(__WXGTK__)

project.h line #552

 public:
    DECLARE_EVENT_TABLE()
//efm5
public:
   bool bStorageDirectorySet;//could use Get/Set() pair here
   void UnattendedCLIAction();
//efm5
};

Project.cpp line #719:

AudacityProject::AudacityProject(wxWindow * parent, wxWindowID id,
[...]
     mMenuClose(false),
     //efm5
     bDoUnattended(false)
     //efm5
{ //this is line #759
//efm5
   //note -- uncomment this to see that cwd has been hijacked by the time we get here as opposed to the real cwd from the cli (see AudacityApp.cpp line #875)
   //wxMessageBox(wxGetCwd());
   //AudacityApp & theApp = wxGetApp();
   //wxMessageBox(theApp.GetCLIpath());
//efm5

the next turns off “save project” if it has no tracks and
also if it is running unattended
Project.cpp line #1862:

   if (event.CanVeto() && (mEmptyCanBeDirty || bHasTracks)) {
      if (mUndoManager.UnsavedChanges()) {
         //efm5
         if (!bDoUnattended) {
            wxString Message = _("Save changes before closing?");
            if( bHasTracks )
            {
               int result = wxMessageBox( Message,
                                         _("Save changes?"),
                                         wxYES_NO | wxCANCEL | wxICON_QUESTION,
                                         this);

               if (result == wxCANCEL || (result == wxYES && !Save()))
               {
                  event.Veto();
                  return;
               }
            }
         }//efm5
      }
   }
   
   ModuleManager::Dispatch(ProjectClosing);

this next adds a new commandline switch to Audacity: -dounattended
it tells Audacity to excercise the new function for unattended operation

AudacityApp.cpp line #1189

        if (!handled && !wxString(wxT("-version")).CmpNoCase(argv[option])) {
            wxPrintf(wxT("Audacity v%s (%s)n"),
                     AUDACITY_VERSION_STRING,
                     (wxUSE_UNICODE ? wxT("Unicode") : wxT("ANSI")));
            exit(0);
         }

        //efm5
         if (!handled && !wxString(wxT("-dounattended")).CmpNoCase(argv[option])) {
            if (!project)
               project = CreateNewAudacityProject();
            project->bDoUnattended = true;
            handled = true;
         }
         //efm5

         if (argv[option][0] == wxT('-') && !handled) {
            wxPrintf(_("Unknown command line option: %sn"), argv[option]);
            exit(0);
         }

         if (!handled)
         {
            if (!project)
            {
               // Create new window for project
               project = CreateNewAudacityProject();
            }
            //efm5
            wxString tempFN(GetCLIpath());
            tempFN.Append(wxT("\"));//must escape NEWLINE to compile
            tempFN.Append(argv[option]);
            project->OpenFile(tempFN);
            if (project->bDoUnattended)
            {
               wxSetWorkingDirectory(GetCLIpath());
               project->UnattendedCLIAction();
            }
            //efm5
            project = NULL; // don't reuse this project for other file
         }
      }                         // for option...

at this point we have a file opened in a project
the next step is to force the project to process (Export) the loaded file
I am going to import an MP3 file and Export it as an M4A
the export will be at my prefered Quality and will be stored in the CWD

Menus.h line# 180:

void OnExport();
//efm5
void OnExport(const wxString & FormatType, const wxString & FileName);
//efm5
void OnExportSelection();

Menus.cpp line# 2944

void AudacityProject::OnExport()
{
   Exporter e;

   e.Process(this, false, 0.0, mTracks->GetEndTime());
}

//efm5
void AudacityProject::OnExport(const wxString & FormatType, const wxString & FileName)
{
   Exporter e;
   e.Process(this, 2, FormatType, FileName, false, 0.0, mTracks->GetEndTime());
}
//efm5

void AudacityProject::OnExportSelection()

Project.h line# 552

public:
   DECLARE_EVENT_TABLE()
//efm5
public:
   bool bDoUnattended;//could use Get/Set() pair here
   void UnattendedCLIAction();
//efm5
};

#endif

here is where we create our workflow
in this case I am exporting the file which was opened because it was passed as a CLI argument
I then tell Audacity to quit

Project.cpp line# 4551 (at end of file)

void AudacityProject::UnattendedCLIAction()
{
   wxFileName fileName(GetFileName());
   wxString tempName(fileName.GetName());
   tempName.Append(wxT(".m4a"));
   OnExport(wxT("M4A"), tempName);
   OnExit();
}

Not having any comments, it appears there is little interest in unattended CLI processing at this time. Still, someone in the future might find this of interest so I will add this update.

The next step in this process is to make Audacity “invisible”; I want it to start with no GUI (no Project window) and never show a Progress Dialog. Here are the code changes (again, line numbers are only approximations as the SVN might have changed, I include a line or two before and after for context and wrap the affected code in
//efm5 start new
//efm5 end new
(note I have added start/end to the pair and “new” for this update):

Project.cpp line #504:

  ModuleManager::Dispatch(ProjectInitialized);
   //efm5 start new
   AudacityApp & theApp = wxGetApp();
   bool dontShowIt = wxGetApp().GetDoUnattended();
   if (!dontShowIt)
      p->Show(true);
   else
      p->Show(false);
   //efm5 end
   return p;

ProgressDialog.cpp line #1195

ProgressDialog::Show(bool show)
{
   if (!show)
   {
   [...]
   }

   //efm5 start new
   bool dontShowIt = wxGetApp().GetDoUnattended();
   return wxDialog::Show(!dontShowIt);
   //efm5 end new
}

I also found that in creating batch files I would occasionally have a typo in the file name resulting in a “failure to open…” MessageBox. These required human intervention so I also changed that:

Project.cpp line #2285 (note–2 affected sections):

   wxString firstLine = wxT("AudacityProject");
   if (!::wxFileExists(fileName)) {
      //efm5 start new
      wxString outputMsg (_("Could not open file: ") + fileName);
      bool doingUnattended = wxGetApp().GetDoUnattended();
      if (!doingUnattended)
         wxMessageBox(outputMsg.c_str(),
                      _("Error opening file"),
                      wxOK | wxCENTRE, this);
      else {
         wcout << outputMsg.mb_str() << endl;
         wxGetApp().SetFileNameError(true);
      //efm5 end new
      }
      return;
   }

   wxFFile *ff = new wxFFile(fileName, wxT("rb"));
   if (!ff->IsOpened()) {
      //efm5 start new
      wxString outputMsg (_("Could not open file: ") + fileName);
      bool doingUnattended = wxGetApp().GetDoUnattended();
      if (!doingUnattended)
         wxMessageBox(outputMsg.c_str(),
                      _("Error opening file"),
                      wxOK | wxCENTRE, this);
      else {
         wcout << outputMsg.mb_str() << endl;
         wxGetApp().SetFileNameError(true);
      //efm5 end new
      }
   }
   char buf[16];

A typical batch file script might look something like:

cd "D:audioKCEA23Aug2010"
"D:audioAudacitySVNwinUnicode ReleaseAudacity.exe" -dounattended "D:audioKCEA23Aug2010Mon Aug 23 0928 2010.mp3" > errors.txt
"D:audioAudacitySVNwinUnicode ReleaseAudacity.exe" -dounattended "D:audioKCEA23Aug2010bogus name.mp3" >> errors.txt
"D:audioAudacitySVNwinUnicode ReleaseAudacity.exe" -dounattended "D:audioKCEA23Aug2010Mon Aug 23 0007 2010.mp3" >> errors.txt

which results in a file “errors.txt” containing:
Could not open file: D:audioKCEA23Aug2010bogus name.mp3
and in an empty saved file with no name as the result of trying to process no data. The next step is to add a check to eliminate this empty file when there is no data to process. Note in the above code that I am now setting a variable in the case of file errors and here is the code which supports this:

AudacityApp.h line #219:

    DECLARE_EVENT_TABLE()
//efm5 start new
// store the current working directory as it exists when Audacity is launched
//(it gets hijacked somewhere -- see Project.cpp line #758
private:
   wxString mCLIpath;
   void SetCLIpath(const wxString & CLIpath) {mCLIpath = CLIpath;};
   bool mbDoUnattended;
   bool mbFileNameError;
public:
   const wxString & GetCLIpath() const {return mCLIpath;};
   bool GetDoUnattended() const {return mbDoUnattended;};
   bool GetFileNameError() const {return mbFileNameError;};
   void SetFileNameError(bool error) {mbFileNameError = error;};
//efm5 end new
};

extern AudacityApp & wxGetApp();

It’s not a commonly sought feature, but it does come up from time to time, and for the people that are looking for this it is often of great importance.

If you have time it might be useful (primarily for the forum “elves”) if you could post a short, non-technical summary so that we can direct people to this thread when it’s appropriate. Is this topic specific to Windows, or would it perhaps be better on the “General Audio Programming” board?

I think that everything I have done would work on Mac and Linux but I can only test Win7. Feel free to move it to where ever you deem appropriate (but kindly PM or email me if you do so so that I may monitor the thread).

short non-technical summary
Detailed specific steps for adding the needed code to Audacity’s source which will allow Audacity to do unattended batch file processing as a background (non-GUI) process. Included is an example of converting many MP3 files into M4A files. Pretty much any series of menu commands can be coded; Nyquist effects would need to have the defaults changed/hard-coded to the user’s choices; Audacity Preferences need to be established in advance. The ability to compile Audacity is required but I am willing to help out with that part.

Feel free to make the above summary a separate post with appropriate links, if desired.

Can you post the compiled version with the added command line processing feature?

http://home.wavecable.com/~edgarmusgrove/index.html#UnattendedAudacity

This is not really intended for the user who is unfamiliar with the Audacity source code and C++ programming! When you run this app from the CLI there will be NO GUI you will have to watch for the command prompt to return to know it is finished. It is really intended for unattended batch processing (I set it running in a batch process with hundreds of file when I go to bed and all is finished in the AM).

If you have a specific need to perform the same effect on large numbers of files, give me some details and I will craft a specific version specifically for you (I will need to know EXACTLY which effects are in your Audacity executable directory’s Plug-ins folder.

I recently ported my changes to Linux (should now work on both Windows & Linux–I have no way to test on Mac) and moved up to revision 11014.

I zipped up the affected source files:
UnA2Linux.zip (151 KB)
I changed the command line switches for clarity and added some new abilities:
-doTo3
-doTo4
-doSilenceProject
-doAmplify

if you call
audacity -doTo3 project.aup
(with the file names fully qualified) the project will open and the audio will be exported as an MP3 (-doTo4 exports as M4A).

[-ed] here are the folder locations for the files in the zip:
srcAudacityApp.cpp
srcAudacityApp.h
srcMenus.cpp
srcMenus.h
srcProject.cpp
srcProject.h
srceffectsAmplify.cpp
srceffectsSilence.h
srceffectsnyquistNyquist.cpp
srcwidgetsProgressDialog.cpp
lib-srclibnyquistnyquistxlispxlread.c

I have posted new executables to play with (windows & Linux):
http://home.wavecable.com/~edgarmusgrove/#CompiledUnattendedAudacity

Not meaning to dismiss this whole command line processing idea, is it not possible to do all this unattended processing using mod-pipe-script? It is even possible (though tortuous) to drive audacity, via mod-pipe-script, from a Windows batch file rather than a perl script. Mod-pipe-script even works in 1.3.13 Beta without hacking the source and recompiling !!!

In this thread on the audacity-users listserv (Feb 2010) http://audacity.238276.n2.nabble.com/mod-script-pipe-module-and-noise-reduction-td4596332.html Gale says:

On this page of the Audacity Manual http://manual.audacityteam.org/index.php?title=Scripting it says:

How to get started with scripting

Audacity 2.0 can only be used with scripting if you compile a separate plug-in module from SVN called mod-script-pipe. You’ll also need the scripting language Perl to try out the examples. We may provide an already compiled mod-script-pipe some time after releasing Audacity 2.0. If you just want to use scripting but don’t want to do the developer steps then you will need to wait for that to be released.

This document refers to the versions of Audacity and mod-script-pipe in SVN as of December 12th 2009. Our intention is to keep this document as up-to-date as possible on the wiki so that it continues to apply to latest builds of Audacity from SVN head.

Building
Ensure you have the latest SVN HEAD version of Audacity built and working. The scripting module, “mod-script-pipe” is in SVN (under lib-src), but is not built automatically.

In MSVC, right-click the mod-script-pipe project and select Build. If all goes to plan, the DLL will be placed in the “modules” directory.
Under GCC, move into the lib-src/mod-script-pipe directory and type “make”. This will build the module and copy the “.so” to the modules directory.

[ToDo] Mac instructions here

Note there is no Mac support documentation and one must still do all the stuff of compiling Audacity then mod-script-pipe to try it.

From what I can tell, Dan Horgan is the author but I don’t see any developer support for mod-script-pipe here on the forum.


@sgparry – do you understand mod-script-pipe well enough to start a new thread and give the forum members a little tutorial and some support?

hi edgard,
i trying to do batch noise reduction on many waves files.
how i can get started with the compiled audacity no gui you trying to develop.
my process flow is this:
1-load wav
2-select part of wav as noise profile(the range as input)
3-call get noise profile
4-apply noise reduction (possible many times)
5-export as wav

any help from you is appreciated.

First and foremost, Audacity is a VERY complicated program! Unless you are a very skilled C++ programmer this project is not for you.

If you are a very skilled C++ programmer, the first thing you need to do is build wxWidgets and Audacity. I think GitHub provides either/both the current alpha code and the most recent release code but it has been quite a while since I have built Audacity. I use a highly modified 2.0.6 code base which is years out of date but the source code with which I am most familiar.

Once you have both wxWidgets & Audacity building and running, I can give you my source code for unattended Audacity. I’m fairly sure that the basic CLI stuff is quite similar. I chose to “turn off” the entire GUI (with the exception of fatal error dialogs) as my monitor is always turned off during this process in doing so saves a few processor cycles. With today’s super-fast super-powerful computers I would probably not bother with all the coding work of turning off the GUI!

I have not figured out any good way of doing noise reduction in the manner to which you allude! The “noise” is rarely pre-definable (neither location/length nor constant-across-files). Remember, a noise profile needs to be at least 1 or 2 seconds in length to be useful. If you are making your own recordings and defined you have constant-across-files noise, is almost always best to identify the problem (almost always in your sound system) and fix that!

Well I am very expert in C++.

I try to achieve this as first step, may be detail me the steps to build wxWidgets and Audacity.

my waves are agent/customer call recorders, for each call I have two waves one Agent side and the other is Customer side,
I am using a VAD(Voice Activity Detection) tool which return list of ranges(start in sec … end in sec) where there is a voice activity on the given wav.
since in single wave there is only one speaker, taking the ranges outside of Voice activities list is noise for me.
according to the process I mention in my last post, the second step is 2-select part of wave as noise profile(the range as input)
this “noise/no voice” 'ranges(take 2/5 second ) from VAD as input to audacity to get the noise profile.
then apply the noise reduction function (maybe call several times) based on that profile and finally export as wav fiie.

If you “only” want to apply noise reduction to a batch of files, it may be better to use a stand-alone noise reduction program rather than Audacity. (a few examples here: https://manual.audacityteam.org/man/sample_workflow_for_lp_digitization.html#alternative)

If you wish to perform automation with the current version of Audacity, it may be worth looking at Audacity’s scripting features: https://manual.audacityteam.org/man/scripting.html

It sounds like you fit the bill experience-wise. First, go to Audacity’s GitHub:

https://github.com/audacity/audacity

login or create an account and either Clone or Download Audacity’s source. There is a first level folder called “win”; it contains a text file: “compile.txt” - the directions are quite explicit and must be followed exactly.

There is a Catch-22/chicken-before-the-egg situation in that to get the compile.txt file you must download the Audacity source but the compile.txt file tells you to download the wxWidgets source from the Audacity GitHub first. Suffice it to say it doesn’t matter if you download the Audacity source first, what does matter is compiling the wxWidgets libraries first.

Once you have wxWidgets built, compile some of the wxWidgets test projects to ensure that libraries are functioning.

After you have wxWidgets compiled and Audacity compiled and running get back to us here and we will see where we go from there.

Probably the only real advantage to making your own personal edits to the Audacity source would be to allow you to create a very large list (depending on your computer, it’s RAM and the version of Audacity the size of the list might be limited - my old computer could handle about 500 sequential launches of Audacity 2.0.6, my current computer has handled over a thousand but it does eventually need to be rebooted) then starting a batch file to process the list overnight.

In the current source code it is called “build.txt” and it can be found here: https://github.com/audacity/audacity/blob/master/win/build.txt

Sorry for the bad information; the last time I checked out the Audacity source was 14 May 2018! I guess now would be a good time to do it again.

HI steve,
thank you for the suggested tool, I am taking a look for them.

I would like to obtain mod-script-pipe module, for the moment audacity tell me ''No module were found"
I have subscribed to the developers mailing list but i didn’t get where to obtain the module.

As you are very experienced with C++, it may be quickest to build Audacity yourself from the source code. The SLNs are set up for Visual Studio 2017 (the free Community version is sufficient). You just have to add the mod-script-pipe option to the build list.

Alternatively, if you post to the developer’s mailing list and say that you are interested in using mod-script-pipe, and as if someone can give you a copy. You will probably get a reply from James, as he is the main developer of this feature.

I am sorry, it was a typo of me not writing NOT, my c++ expertise is basic.
I took a look in audacity src folder in github. it seem for me so complex to compile.
I find it easier for me to use audacity scripting as you share me the wiki page of that.