possible to batch edit 17,000 slightly different .mp3 files?

Anything you think is missing, or needs enhancement, goes here.
Forum rules
If you require help using Audacity, please post on the forum board relevant to your operating system:
Windows
Mac OS X
GNU/Linux and Unix-like
aud-dude
Posts: 5
Joined: Mon Mar 26, 2018 7:27 pm
Operating System: OS X 10.11 El Capitan or later (macOS)

possible to batch edit 17,000 slightly different .mp3 files?

Post by aud-dude » Mon Mar 26, 2018 8:15 pm

I have 17,000 .mp3 files from a dictionary project.

The first file starts off with a recording of someone saying, "One." What follows after that is the part of the audio recording that I want to keep.
The second file starts off with someone saying, "Two," and then the stuff I want keep.
Etc., etc., etc. through the 17,000th file.

Of course after each number is said, there is a brief silent pause.

It will not work to just delete the first second (for example) of each file, because that would be too long for some files (it doesn’t take long to say, “one”) and too short for others (it would take longer to say, “16,783,” for example).

Can Audacity somehow be set up to await that brief pause, and then split that file into two parts: (1) the part where the number is said, which I'd be glad to delete, (2) the part with the rest of the file that I want to keep?


Helllllllp me please! I'm an Audacity rookie. You will not offend me by being super specific and detailed in your replies and directions. :D


And thank you!

kozikowski
Forum Staff
Posts: 40449
Joined: Thu Aug 02, 2007 5:57 pm
Operating System: OS X 10.9 Mavericks
Location: Los Angeles

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by kozikowski » Mon Mar 26, 2018 8:25 pm

I think we can do the silent sense, but I don't think we can make the Chains/Batch system look for the marks, and then only use the first mark.

Someone better at Batch/Chains may post.

Also, there may be a limit to the number of files. Audacity makes copies of The Whole Job every time you take an action so it can perform UNDO. That process is going to get enormous after a while.

Koz

aud-dude
Posts: 5
Joined: Mon Mar 26, 2018 7:27 pm
Operating System: OS X 10.11 El Capitan or later (macOS)

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by aud-dude » Tue Mar 27, 2018 9:44 am

Thanks @Koz, any wisdom how I might engage someone who is "better at "Batch/Chains" with this question?

The file sizes are about 350kb each, relatively small, but we could do them in groups to overcome the file size challenge.

kozikowski
Forum Staff
Posts: 40449
Joined: Thu Aug 02, 2007 5:57 pm
Operating System: OS X 10.9 Mavericks
Location: Los Angeles

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by kozikowski » Tue Mar 27, 2018 10:35 am

how I might engage someone
No, and that's not the only problem you're going to have. You violated a process recommendation—never do production in MP3. Nasty things can happen.

Audacity doesn't edit MP3. It converts MP3s to its internal editing format and then makes a new MP3 when it's done, so anything you do is going to have double the MP3 compression sound distortion. You can avoid the increase by exporting the finished works as WAV and never making another MP3, or you can get less damage by making very high quality MP3s. Both of those will give you much higher file sizes than you think.

You may be better off with one of the pure MP3 editors that don't take the sound apart and put it back together.

Scroll down to Other MP3 Editing Tools.

http://wiki.audacityteam.org/wiki/MP3#re-encode

Koz

waxcylinder
Forum Staff
Posts: 9435
Joined: Tue Jul 31, 2007 11:03 am
Operating System: Windows 10
Location: Manchester, UK

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by waxcylinder » Tue Mar 27, 2018 10:37 am

I certainly don't think you can do it with Chains.

But for the next release 2.3.0 we are changing Chains to Macros with added functionality (more commands that you can put in a Macro - and other stuff)

We recommend never processing more than 500 files at a time in a Chain or Macro (smaller numbers may be even better)

You can't do it right now with Macros AFAICT, but I am in discussion with the developer who is handling Macros - we may need to add a command (or two) if he is prepared to do that - I'll let you know ...

The Export process in Macros can be a bit flaky right now - but we should have that fixed in the next month - so can you wait a bit


First let me ask you a few questions about your files:

1) are there other 1-plus second silences in each file,

2) is one second the minimum length of silence between the spoken number and the required audio,

3) what output format are you expecting (audio files, Audacity Projects),

4) are you expecting to write back, overwrite, the existing input file(s) probably not a good idea,

5) can I assume that you would make a safety backup copy of all thes files vefore exposing tem to Macro processing,

6) would you be prepared to take the (managed) risk of using Beta software for this ?

Peter
________________________________________FOR INSTANT HELP: (Click on Link below)
* * * * * FAQ * * * * * Tutorials * * * * * Audacity Manual * * * * * Audacity Wiki * * * * *

steve
Site Admin
Posts: 47294
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu
Contact:

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by steve » Tue Mar 27, 2018 10:38 am

Audacity's Chains provide a simple "list" processing feature. That is, you have a list of commands, and the Chain works down the list from top to bottom. There is no mechanism for logical operations (such as conditional branches).

Also, Chains should not generally be used for more than a couple of hundred files at a time. Better to split your collection of 17000 into smaller groups. Say, 34 groups of 500 files.

Possible options depend on budget. Is this a commercial job, is there a budget?
I'm also wondering why you have 17000 specially prepared files, before having a clear plan of how to process them in the way that you need. What's the job? What's the big picture?

Do you (or your team) have any programming experience?
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)

kozikowski
Forum Staff
Posts: 40449
Joined: Thu Aug 02, 2007 5:57 pm
Operating System: OS X 10.9 Mavericks
Location: Los Angeles

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by kozikowski » Tue Mar 27, 2018 10:41 am


waxcylinder
Forum Staff
Posts: 9435
Joined: Tue Jul 31, 2007 11:03 am
Operating System: Windows 10
Location: Manchester, UK

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by waxcylinder » Tue Mar 27, 2018 2:40 pm

Not possible to do with Chains ...

However with 2.3.0 alpha Macros some of the added commands help us here

First the good news.
This little Macro does the job:
SelectAll:
SilenceFinder:labelbeforedur="0.3" sil-dur="1" sil-lev="26"
SelectAll:
SplitLabels:
SelectAll:
CursProjectStart:
SelNextClip:
Delete:
ExportWav:

The silence finder command will find all silences of longer than 2.0 seconds abd above the thrfeshold - those are settable parameters
It will then place a label 0.3 second before the non-silence starts (settable parameter)
we then place split markers in the audio at all the label positions (that's the SplitLabels command)
We then move the cursor back to home T=0
and then select all up to the next clip (i.e. from T=0 to 0.3 seconds before your required audio_
We than delete the section
final step is to export

Right now I have the export set to WAV - this is for a couple of reasons.
1) As you input is MP3 re=exporting to MP3 will increase the compression damage
2) as it stands tight now exporting to MP3 would just overwrite your existing source MP3 with no warning (this is one of the wrinkles we want to fix)

Now the bad news
The SplitLabels command has a bug which we only discovered when I was setting up and testing this Macro.
Basically it pops an error when you try to run it the first time (or with out a prexisting label track) with a selection apparently I'm told)

This of course makes it unusable as a batch-process.

I have managed to bench-test the Macro - but on the open project and not a set of files - I have to run it twice, and the second time it places the split marker(s) - I will see if I can find a better workaround, but I'm not hopeful.

I will shortly be logging the bug.

In the meantime if you want to be bold and test this on one of your files using a nightly alpha you would be welcome. You can get the latest Mac nightlies from here: https://www.fosshub.com/Audacity-Mac-Nightlies.html

I have attached the actual .txt file with the Macro - you would need to download 2.3.0 and add this to your Macros folder.

The draft documentation for the Macros can be found here: https://alphamanual.audacityteam.org/man/Macros

And thanks to James Crook (Macros developer) for helping me with what looks on the surface a very little bit of Macro coding - but it's only easy when you know how 8-)

Cheers,
Peter
Attachments
silence remove.txt
(165 Bytes) Downloaded 23 times
________________________________________FOR INSTANT HELP: (Click on Link below)
* * * * * FAQ * * * * * Tutorials * * * * * Audacity Manual * * * * * Audacity Wiki * * * * *

aud-dude
Posts: 5
Joined: Mon Mar 26, 2018 7:27 pm
Operating System: OS X 10.11 El Capitan or later (macOS)

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by aud-dude » Tue Mar 27, 2018 2:44 pm

waxcylinder wrote:I certainly don't think you can do it with Chains.

1) are there other 1-plus second silences in each file,

Yes. Ouch.

2) is one second the minimum length of silence between the spoken number and the required audio,

I think it's actually less than a second.

3) what output format are you expecting (audio files, Audacity Projects),

Something pretty universal. MP3 is fine.

4) are you expecting to write back, overwrite, the existing input file(s) probably not a good idea,

I'm flexible.

5) can I assume that you would make a safety backup copy of all thes files vefore exposing tem to Macro processing,

Yes. It's backed up.

6) would you be prepared to take the (managed) risk of using Beta software for this ?

I'm up for any experiment, and would even find some resources to find someone who would help me.

Peter
Thanks!

waxcylinder
Forum Staff
Posts: 9435
Joined: Tue Jul 31, 2007 11:03 am
Operating System: Windows 10
Location: Manchester, UK

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by waxcylinder » Tue Mar 27, 2018 2:56 pm

And now that James, the developer, understands the underlying issue for the bug - he's given mee a workaround.

The Macro now reads:
SelectAll:
SilenceFinder:labelbeforedur="0.3" sil-dur="1" sil-lev="26"
SelectAll:
ShowExtraMenus:
ShowExtraMenus:
SplitLabels:
SelectAll:
CursProjectStart:
SelNextClip:
Delete:
ExportWav:

Note that we now have two additional extraneous commands of ShowExtraMenus - this is enough to "fool" Audacity into working the SplitLabels command

If you're interested in the technical details - James wrote:
Add Extra Menus (on/off) TWICE immediately before the Label Split (SplitLabels) command.
That causes the menus and their flags to be regenerated. And then the command will work because the command is not greyed out in the menus anymore after the greying-out flags are re calculated!


I've attached an updated copy of the .txt file for the Macro

Peter.

P.S. this has been a useful QA test of the new Macros for me 8-)
Attachments
silence remove.txt
(199 Bytes) Downloaded 23 times
________________________________________FOR INSTANT HELP: (Click on Link below)
* * * * * FAQ * * * * * Tutorials * * * * * Audacity Manual * * * * * Audacity Wiki * * * * *

Post Reply