possible to batch edit 17,000 slightly different .mp3 files?

Anything you think is missing, or needs enhancement, goes here.
Forum rules
If you require help using Audacity, please post on the forum board relevant to your operating system:
Windows
Mac OS X
GNU/Linux and Unix-like
waxcylinder
Forum Staff
Posts: 9436
Joined: Tue Jul 31, 2007 11:03 am
Operating System: Windows 10
Location: Manchester, UK

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by waxcylinder » Tue Mar 27, 2018 3:00 pm

BTW this Macro will only work properly if you don't have a silence if 1.0 seconds or greater BEFORE the spoken number.

This Macro (as currently parameterized) just deletes the first block of silence of grater that 1.0 (parameterizable) seconds that it finds.

Also you may need to tweak the "silence" level depending on how "silent" you "silence" is :geek:

Peter.
________________________________________FOR INSTANT HELP: (Click on Link below)
* * * * * FAQ * * * * * Tutorials * * * * * Audacity Manual * * * * * Audacity Wiki * * * * *

aud-dude
Posts: 5
Joined: Mon Mar 26, 2018 7:27 pm
Operating System: OS X 10.11 El Capitan or later (macOS)

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by aud-dude » Tue Mar 27, 2018 3:07 pm

Thanks so much, all!
~~~~~
1) are there other 1-plus second silences in each file,

There would be several pauses per recording. I think almost all of the pauses would be less than a second.

That would be no problem this Macro only removes the first "silence" that it finds

2) is one second the minimum length of silence between the spoken number and the required audio,

No, of course I've not listened to all 17,000 of the files, but think the silences would be shorter than a full second.

In which case we would need to tweak down that 1 second minimum silence length

but we have to make sure that whatever we set it at exceeds ant "silence" that may be before the number


3) what output format are you expecting (audio files, Audacity Projects),


I'm flexible, but something pretty standard would be best. mp3?
You'll double the compression damage - but you're not making hi-fi audio

4) are you expecting to write back, overwrite, the existing input file(s) probably not a good idea,

I'm flexible here, too.
Right now if you have mp3 as input and we change the Macro to export to MP3 then overwrites will occur - James is gonna wortk later to fix this.

5) can I assume that you would make a safety backup copy of all these files vefore exposing them to Macro processing,

Yes, I have redundant file copies, so we can be aggressive in experimenting.
Good - so not entiely naive after all ...

6) would you be prepared to take the (managed) risk of using Beta software for this ?

In theory, yes. In practice, I'm a total novice to Audacity.
I like bold people :D
~~~~~~~~~

Doing groups of 500 or whatever would be fine.

The big picture is that I'm helping make a dictionary for a language. Along the way I hired a linguist to help. There ended up being about twice as many words as anticipated. We did not realize that the numbers would be added to the audio files. While it's hard for me to imagine ever breaking even financially on this project, I have used money to get us this far, and would be open to spending some more for a great and quick solution.

I have almost no programming or Audacity experience.

So some hand-holding may be necessary - we can help.

I'll post some getting-started instructions in a fresh post


Peter

Thanks Peter, Steve, and James!

Edgar
Forum Crew
Posts: 1516
Joined: Thu Sep 03, 2009 9:13 pm
Operating System: Windows 7
Contact:

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by Edgar » Tue Mar 27, 2018 3:20 pm

waxcylinder wrote:BTW this Macro will only work properly if you don't have a silence if 1.0 seconds or greater BEFORE the spoken number.
I've been following along without interrupting, but… Does the current Alpha have some form of a Trim (front/back/both) command/Effect which could be run before finding silences?
-Edgar
compiling Audacity daily
64-bit Windows 7

waxcylinder
Forum Staff
Posts: 9436
Joined: Tue Jul 31, 2007 11:03 am
Operating System: Windows 10
Location: Manchester, UK

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by waxcylinder » Tue Mar 27, 2018 3:25 pm

________________________________________FOR INSTANT HELP: (Click on Link below)
* * * * * FAQ * * * * * Tutorials * * * * * Audacity Manual * * * * * Audacity Wiki * * * * *

waxcylinder
Forum Staff
Posts: 9436
Joined: Tue Jul 31, 2007 11:03 am
Operating System: Windows 10
Location: Manchester, UK

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by waxcylinder » Tue Mar 27, 2018 3:59 pm

1. In Finder rename your current Audacity folder in the Applications folder to say "Audacity Released".
That will preserve you existing Audacity installation

2) Go to and download the latest Mac alpha DMG

3) Run the DMG and drag the app over into the Applications folder (do NOT run from the DMG)

4) copy the attached Macro silence remove.txt file from here to somewhere on your Mac (desktop is fine)

5) Copy this file into the "Macros" folder - on Mac this is in the following folder: ~/Library/Application Support/audacity/
note the "~" (tilde) - that's your user space not system space

6) Go to the Applications folder and right click on the new Audacity.app

7) This is unsigned so on first usage you have to permit it to run (it'll just run on subsequent uses)

8) Import one of you files into Audacity - have a look to make sure that there is less than one second before the spoken number and more than one second after the spoken number

Now for the good bit 8-)

9) use Tools >Macros.. - you will get the Manage Macros dialog

10) in the left hand Select Macro box you should see silence remove

11) left-click on that - it will get highlighted and the Macro commands will show in the right hand pane

12) Now click on the Project button to the right of the "Apply Macro to:"

13) This will run the macro

14) because we're running non-batch on the project it should ask you where you want to store the output mp3 file

15) Report back ;)

Peter

I need a lie-down in a darkened room ...
________________________________________FOR INSTANT HELP: (Click on Link below)
* * * * * FAQ * * * * * Tutorials * * * * * Audacity Manual * * * * * Audacity Wiki * * * * *

waxcylinder
Forum Staff
Posts: 9436
Joined: Tue Jul 31, 2007 11:03 am
Operating System: Windows 10
Location: Manchester, UK

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by waxcylinder » Tue Mar 27, 2018 4:04 pm

Edgar wrote:
waxcylinder wrote:BTW this Macro will only work properly if you don't have a silence if 1.0 seconds or greater BEFORE the spoken number.
I've been following along without interrupting, but… Does the current Alpha have some form of a Trim (front/back/both) command/Effect which could be run before finding silences?
Hmmm - that's a thought Ed - I'll have think about that


Peter
________________________________________FOR INSTANT HELP: (Click on Link below)
* * * * * FAQ * * * * * Tutorials * * * * * Audacity Manual * * * * * Audacity Wiki * * * * *

waxcylinder
Forum Staff
Posts: 9436
Joined: Tue Jul 31, 2007 11:03 am
Operating System: Windows 10
Location: Manchester, UK

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by waxcylinder » Tue Mar 27, 2018 4:12 pm

waxcylinder wrote:
Edgar wrote:
waxcylinder wrote:BTW this Macro will only work properly if you don't have a silence if 1.0 seconds or greater BEFORE the spoken number.
I've been following along without interrupting, but… Does the current Alpha have some form of a Trim (front/back/both) command/Effect which could be run before finding silences?
Hmmm - that's a thought Ed - I'll have think about that

Peter
I can't see one - only the old Trim Audio command.

Are you thinking, Ed, Of James' pre-supplied "Fade Ends" Macro - which does Fades not Trims.

Peter
________________________________________FOR INSTANT HELP: (Click on Link below)
* * * * * FAQ * * * * * Tutorials * * * * * Audacity Manual * * * * * Audacity Wiki * * * * *

steve
Site Admin
Posts: 47297
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu
Contact:

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by steve » Tue Mar 27, 2018 5:06 pm

Edgar wrote:Does the current Alpha have some form of a Trim (front/back/both) command/Effect which could be run before finding silences?
You can trim a specified number of seconds from each end.
Here's an example macro script:

Code: Select all

SelectTime:End="2" Start="0"
Delete:
SelectTime:End="2" RelativeTo="Project End" Start="0"
Delete:
For more complex "arithmetic" operations, a proper scripting language (such as Python or Nyquist) should be used rather than macros. Like the current "Chains", Macros are intended for applying a list of command (the difference being that Macros support many more commands than Chains did). The commands available to Macros will also be available to external scripting languages, and eventually also to Nyquist. (Nyquist support will be added later as there are issues with reentrancy that still need to be resolved).
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)

waxcylinder
Forum Staff
Posts: 9436
Joined: Tue Jul 31, 2007 11:03 am
Operating System: Windows 10
Location: Manchester, UK

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by waxcylinder » Tue Mar 27, 2018 5:25 pm

steve wrote:You can trim a specified number of seconds from each end
useful - but no help to the original poster here - they don't know how long an possible front-end silence is but they do know there is a "silence" between the initial spoken number and the audio they want to retain.

WC
________________________________________FOR INSTANT HELP: (Click on Link below)
* * * * * FAQ * * * * * Tutorials * * * * * Audacity Manual * * * * * Audacity Wiki * * * * *

Edgar
Forum Crew
Posts: 1516
Joined: Thu Sep 03, 2009 9:13 pm
Operating System: Windows 7
Contact:

Re: possible to batch edit 17,000 slightly different .mp3 fi

Post by Edgar » Tue Mar 27, 2018 7:26 pm

waxcylinder wrote:
Edgar wrote:Does the current Alpha have some form of a Trim (front/back/both) command/Effect which could be run before finding silences?
I can't see one - only the old Trim Audio command.
Are you thinking, Ed, Of James' pre-supplied "Fade Ends" Macro - which does Fades not Trims.
Stevethefiddle wrote me a custom "TrimSilence" Nyquist Effect:

Code: Select all

;nyquist plug-in
;version 3
;type process
;categories "http://lv2plug.in/ns/lv2core/#UtilityPlugin"
;name "Trim Silence..."
;action "Trimming..."
;info "by Steve Daulton (www.easyspacepro.com). Released under GPL v2.\n\nTrims silence from the beginning and end of the selection.\n"

;control thresh "Silence Threshold (dB)" real "" -38 -100 0

;; TrimSilence.ny by Steve Daulton. Aug 2011.
;; Updated 24 Sept 2012.
;; Released under terms of the GNU General Public License version 2:
;; http://www.gnu.org/licenses/old-licenses/gpl-2.0.html
;; Requires Audacity 1.3.8 or later.

;; RAM USAGE:
;; This plug-in requires the audio to be loaded into RAM.
;; If there is insufficient free RAM Audacity may freeze or crash.
;; The line below limits RAM usage to 1.0 GB (about 47 minutes for
;; a stereo track at 44.1 kHz)
;; If your computer has more than 1GB of physical RAM available, the
;; limit may be increased.

(setq RAM-Limit 2.5) ; RAM limit in GB

; convert threhold to linear
(setq thresh (db-to-linear (min 0 thresh)))

;; Limit of duration in seconds
(setq limit
  (/ (* ram-limit 1000000000)
    (* 4.0 *sound-srate*)))
(when (arrayp s)(setq limit (/ limit 2.0)))

;;; modulo
(defun mod (x y)
  (setq y (float y))
  (round (* y
    (- (/ x y)
      (truncate (/ x y))))))
  
;;; convert to hh:mm:ss
(defun to-hhmmss (seconds)
  (let* ((hh (truncate (/ seconds 3600)))
        (mm (truncate (/ (mod seconds 3600) 60)))
        (ss (mod seconds 60)))
    (format nil "~ah:~am:~as" hh mm ss)))

;;; convert to mono and limit sample rate
(defun convert (sig ratio)
  (if (arrayp s)
      (snd-avg 
        (s-max (snd-abs (aref sig 0))
          (snd-abs (aref sig 1)))
        ratio ratio op-peak)
      (snd-avg sig ratio ratio op-peak)))

;;; find silences
(defun find-sil (sig &aux (start 0)(end 0))
  (do ((new (snd-fetch sig) (snd-fetch sig))
       (flag 0))
      ((not new))
    (if (= flag 0)
        ;; count initial silence
        (if (<= new thresh)
            (setq start (1+ start))
            (setq flag 1))
        ;; count final silence
        (if (<= new thresh)
            (setq end (1+ end))
            (setq end 0))))
  (list start end))


(if (< len (* limit *sound-srate*)) ;max length in samples
  (let* ((start 0)
         (end 0)
         (flag 0)
         ;; ratio provides tighter trimming for short selections
         ;; while maintaining reasonable speed for long selections
         (ratio (max 10 (min 200 (round (/ len 100000.0)))))
         (my-srate (/ *sound-srate* ratio))
         (mysound (convert s ratio)))

    ;loop through samples and mark start and end
    (setf result (find-sil mysound))

    (let ((start (/ (first result) my-srate))
          (end (- (get-duration 1)(/ (second result) my-srate))))
      ;; ensure at least 1 sample remains
      (if (>= start (get-duration 1))
        (setq start (/ (1- len) *sound-srate*)))
      ; trim
      (multichan-expand #'extract-abs start end (cue s))))

  ;; OR print error message
  (format nil "Error.\nMax RAM usage by Trim Silence is set to ~a GB.~%This allows a maximum duration ~
              for a ~a~%track at ~a Hz of ~a.~%Selected track is ~a.~%"
              RAM-limit
              (if (arrayp s) "stereo" "mono")
              (round *sound-srate*)
              (to-hhmmss limit)
              (to-hhmmss (get-duration 1))))
Pay attention to the note on RAM usage:

Code: Select all

;; RAM USAGE:
;; This plug-in requires the audio to be loaded into RAM.
;; If there is insufficient free RAM Audacity may freeze or crash.
;; The line below limits RAM usage to 1.0 GB (about 47 minutes for
;; a stereo track at 44.1 kHz)
;; If your computer has more than 1GB of physical RAM available, the
;; limit may be increased.

(setq RAM-Limit 2.5) ; RAM limit in GB
As you can see I have changed this from 1.0 to 2.5 as I have plenty of RAM. However, I have also written a "built-in" Effect in C++; it is much faster, a bit smarter, and allows for three options: trim silence off the front, trim silence off the back & trim silence off both the front and the back. It also gives direct access to the amplitude to consider as "silence". There is also a checkbox for "Assume a short track"; if checked, it loads the entire track into RAM for processing; otherwise it bites a chunk off the front and/or back (one at a time as needed) does a little checking to verify that you got a good sample (e.g. for front has something that is not "silence" as the final few samples, for back as something that is not "silence" as the first few samples) and if not bites of bigger chunks and retests. The GUI needs a little TLC as statically and looks like this:
trim.png
Audacity allows you the choice to run one effect (Normalize) every time you import; I have extended that functionality:
prefs.png
-Edgar
compiling Audacity daily
64-bit Windows 7

Post Reply