Dealing with thousands of tracks

[tlm]

While using Audacity to manipulate audio files for language learning, I’ve often imported hundreds or even thousands of small carefully named .wav files as tracks which are then aligned end-to-end before mixing and rendering them.

Recently I tried to do this with about 8000 .wav files and (not surprisingly) Audacity crashed (after many hours). The image below shows Audacity is using over 21GB while not responding.

I’m running Audaicty 3.3.3 using a MacAir 2020, M1 with 16 GB ram.

My question is, is there a better way of doing what I’m trying to do?

Here are the details of what I do: I start with a source audio file that has words in one language (language1) followed by a gap of silence (gap1) and then the translation (language2) followed by a gap of silence (gap2), etc. So, the pattern is:

language1 - gap1 - language2 - gap2 - etc…

What I want to produce are different files such as:

language1 - differentGap1 - language2 - differentGap2 - etc…
or
language2 - differentGap2 - language1 - differentGap1 - etc…

Varying the silence gap length gives the language learner more or less time to translate in their head depending on their proficiency.

Reversing the language order is also useful so one can learn to translate in both directions.

My approach is to use the Label Sounds plugin to label and then export all those “language1” and “language2” sounds as .wav files.

The length of the silence gaps should be some multiple of the preceding sound files. So what I do is produce silent versions of all the “language1” and “language2” files and then use some multiple of those silence files after each sound.

I then label all those files very carefully, put them into the same folder and drag all of them into Audacity. Once finished loading, I select them all, use Tracks → Align Tracks End to End and then Tracks → Mix and Render to give the final desired audio.

When it crashed recently I had about 1000 files for each language, plus 3 silences behind each language, amounting to the 8000 files that I dragged into Audacity and waited… for it to crash. Before that Audacity finished 3000 of them and (thanks to Audacity’s recovery features) I was able to mix and render to produce a good partial product file.

[billw58]

I think your solution is in your last paragraph.

Process the files in chunks, say 1000 or so files at a time. Mix and render the chunk, then move on to the next.

Once you’ve got all the chunks mixed and rendered, import those chunks and proceed to align and mix.

– Bill

[tlm]

Thanks @billw58 for reading through my detailed post. Yes, I have used that approach and it does work. Of course I was hoping someone might have an entirely better way of doing it.

[christop]

If you’re comfortable with a bit of programming/scripting, you could use something like SoX which calls itself “the Swiss Army knife of sound processing programs”. It’s a command-line program and can have a bit of a steep learning curve for some of its effects, but concatenating multiple audio files is fairly straightforward.

[tlm]

Thank you @christop - I do have programming experience and I am very interested in audio processing. I once tried to figure out how Nyquist plug-ins work but I found it was beyond me. So you can see that I have my limits. On the other hand, if reasonable documentation is available, I am a good student. So, if you could point me in the direction of any good tutorials/documentation, I would appreciate it. Of course I can also search on my own

[steve]

christop’s suggestion is a good one.

SoX is a command line app, so you can program with it using almost any language. You can create a loop in Bash, PowerShell, Python, Perl, Basic or some other language, and in that loop make a system call to run SoX with appropriate arguments.

The SoX documentation is here: SoX - Sound eXchange | Documentation

There are also several examples of using SoX in this forum. Try searching for “SoX” using the search box at the top of https://forum.audacityteam.org/

[tlm]

I ended up solving the problem using ffmpeg instead of SoX when I realized that all I was doing was concatenating files. Audacity is great at generating the thousands of files that I need, but (at least with the steps I was using) it is unsuited to concatenate those files… Or I should say, it can’t handle the step in which I drag those 8000+ files into an Audacity window. Once they are in, Audacity works relatively quickly.

But the speed at which ffmpeg concatenated my 8000+ files was quite something: about a minute. Here are the commands I used, based on the instructions on this site:

First, I navigated to the folder with all the 8000+ files, named so that they would be in the order I wanted them concatenated. Then, I had to execute this Terminal command to produce a listing of all the files to be concatenated:

for f in *.wav; do echo "file '$f'" >> mylist.txt; done

then this ffmpeg command quickly concatenated them:

ffmpeg -f concat -safe 0 -i mylist.txt -c copy output.wav

Thanks to all for the help. I knew there had to be a better way.

[christop]

ffmpeg works too.

I think the equivalent SoX command would be sox *.wav output.wav, which seems easier to me than having to write a list of files to a text file first (though that does give you flexibility in which files to include and their order).