Some On Demand Loading questions

Audacity Help Forum Windows
1

I’m experimenting with using On Demand Loading because I have some very large projects to work on, and I have a few questions about the implications of using this mode.

  • It’s excellent that it creates the waveform in the background because that lets me get on with the job of importing the next file. But once it’s been calculated, does it do any further background calculations when I’m zooming and panning around? It doesn’t seem to, but I sometimes have to use an older, slow computer and I was hoping it would.
  • It looks like the background calculations don’t apply to spectrograms. It seems like it has to finish calculating the waveform first before I can change to spectrogram view, although it’s great that I can zoom in while it’s still calculating, and it then quickly calculates for that section and then lets me change the view. Are there any plans to allow the view to be changed before the waveform is calculated?
  • When I save the project, the data folder is much smaller than if I hadn’t used On Demand Loading, but there are still a lot of files in the data folder. What are they?
  • If I reopen the project, it displays the waveforms very quickly. Does that mean the waveform has been saved?
  • The initial saving of the project takes a long time, presumably because it’s saving the waveform. Is there a way to skip that so that the initial save is quick?
  • If I check dependencies, it gives me the option to save some or all of the wav files. If I save them all, is my project now identical to the project I would have had if I had initially imported and copied the audio? If I save just some of them, does that mean On Demand Loading only applies to the other tracks?
  • What happens if I want to move the project and all its files to another drive? Is there any solution other than editing the aup file to fix the aliasfile locations? Is there a way for these to be relative paths?
  • Is there anything else about On Demand Loading that I need to know? I understand that I have to be careful not to move the wav files, and I also assume any edits I do will be performed on the original files.
2

Another question. I’ve got over 20 hours of recordings imported into two projects - one using On Demand Loading, the other copied into the project. The copied audio project is always slow to open (minutes). The On Demand project is always much faster, but is sometimes very fast (e.g. just a couple of seconds compared to perhaps 10 or 20 seconds).

If the copied audio project has already been set up, why so slow to load?

And why is the On Demand project sometimes faster than other times?

Is there any log kept of project opening events? If so then I’d like to use it to time these things.

3
  • When I save the project, the data folder is much smaller than if I hadn’t used On Demand Loading, but there are still a lot of files in the data folder. What are they?

The waveform generation files and any difference between the original work and the new work. I’m sure you discovered Audacity does not save UNDO.

  • If I check dependencies, it gives me the option to save some or all of the wav files. If I save them all, is my project now identical to the project I would have had if I had initially imported and copied the audio?

If you give Audacity “personal copies” of all the music, then yes that’s the same show had you done that at the beginning.

  • What happens if I want to move the project and all its files to another drive? Is there any solution other than editing the aup file to fix the aliasfile locations? Is there a way for these to be relative paths?

I don’t think so. That’s the down side of trying to move a Project and it’s the main reason to not use relative files. The files and directories in the AUP file must match reality exactly for Audacity to find everything. I suppose you could push everything into a /music folder in the same folder as the project and just tell Audacity “./music/StarSpangledBanner.wav” instead of a fully qualified path. Try it. Let us know what happens.

Koz

4

Again, the Project is the waveform files and any difference between the show and the files. If there is no difference, the show will load in no time.

Audacity has to load all your music into memory to do anything with it in non-reference, so you are looking at the time it takes your spinning drives to manage all that work. If you have short memory, the machine has to swap out and it can take very serious time.

If you had SSDs, the work would flash by. I didn’t realize the difference until the first time I had to do production on a friend’s “spinning rust” drives. It felt like I could go out for a run on the beach while the machine did anything. Similar machine, but I had SSDs. Very Highly recommended for serious production.

Koz

5

If you’re on XP, you should take steps to make sure your drives and roomy, error checked, and defragmented. Nothing slows a machine down like having to sweep up music like shattered debris from thousands of different places on the drive. If you’re on more modern machines, make sure you’re not filling up any drive the machine can see.

Koz

6

What do you mean by “the show”?

Audacity has to load all your music into memory to do anything with it in non-reference, so you are looking at the time it takes your spinning drives to manage all that work. If you have short memory, the machine has to swap out and it can take very serious time.

What do you mean by “non-reference”?

If you had SSDs, the work would flash by. I didn’t realize the difference until the first time I had to do production on a friend’s “spinning rust” drives. It felt like I could go out for a run on the beach while the machine did anything. Similar machine, but I had SSDs. Very Highly recommended for serious production.

I didn’t realise that, I assumed SSD was best for rarely changing data like programs. Can you recommend any brands and models, or any to avoid?

And would the fast access times still have any benefit if connected via USB 3 or even USB 2? I have one USB 2 machine, and it would be good if it would help there too.

7

I’m on win 7, but I’d better check if defragmentation is running. Plenty of room left on all drives.

I expect speed problems on a large project like this, but I’m intrigued by the speed variations when I only open, close, reopen, etc. I.e no changes before reopening.

8

Do you mean that if I edit the tracks then the files don’t actually get changed? How are the changes stored? As editing commands, or as waveforms to replace the originals? If the latter then does that mean that I edit enough parts of the tracks then I might as well not be using On demand mode?

9

A big question, so I’ll try and break it down into points.
Also, see here for an overview of the technology:
http://manual.audacityteam.org/o/man/audacity_projects.html
in particular, this section: Audacity Manual


Audacity will recalculate the waveform whenever the audio data is changed.
Many people don’t notice but this is a significant difference between Audacity and most DAW applications. If, for example, you apply Equalization to a track in Audacity, the audio data is changed and the waveform redrawn. In most DAWs, applying Equalization makes no difference to the drawn waveform.

The “background drawing” effect in “On Demand” only applies when initially loading the audio. During later operations the waveform and redrawing is done when required whether “OD” is enabled or not.

It looks like the background calculations don’t apply to spectrograms.

Correct. The “OD” waveform calculation applies only to the waveform view.

It seems like it has to finish calculating the waveform first before I can change to spectrogram view, although it’s great that I can zoom in while it’s still calculating, and it then quickly calculates for that section and then lets me change the view. Are there any plans to allow the view to be changed before the waveform is calculated?

There is already quite a lot that you can do while the “OD stripes” are displayed.
Try opening a large file with OD, and immediately click in the middle of the track and start playing. Provided that the computer is fast enough, the track will start playing and OD will prioritise this part of the waveform to draw first.
I rarely use OD myself as most of my projects are quite short, but it is quite impressive just how much OD will allow you to do while “waiting” for the waveform.

  • When I save the project, the data folder is much smaller than if I hadn’t used On Demand Loading, but there are still a lot of files in the data folder. What are they?

Probably not technically precise, but I think of them as “place holders”.
Each of those files represents a “block” of audio data. With “normal” loading, a file is copied into the project as a load of “blocks” of audio and each block is a short section of one audio channel. With OD, the audio data is not copied during loading, but Audacity still needs its “block files”, so small “place holder” files are created to represent each audio data block. When you process the audio, these blocks are replaced by real audio data.

  • If I reopen the project, it displays the waveforms very quickly. Does that mean the waveform has been saved?

Audacity uses several optimisations to reduce the time taken to draw the waveform. The project data file include a kind of “snapshot” of the waveform. The details of the optimisations are quite complicated. Is it enough to just say that it is “clever”? :wink:

  • The initial saving of the project takes a long time, presumably because it’s saving the waveform. Is there a way to skip that so that the initial save is quick?

On saving, there is a lot of data that needs to be moved.
Temporary data in RAM is deleted. Temporary files are deleted (there can be several GB of temporary files for big projects). Much of the time spent saving projects is spent on cleaning up old temporary data.

  • If I check dependencies, it gives me the option to save some or all of the wav files. If I save them all, is my project now identical to the project I would have had if I had initially imported and copied the audio?

Yes.

If I save just some of them, does that mean On Demand Loading only applies to the other tracks?

Yes.

  • What happens if I want to move the project and all its files to another drive? Is there any solution other than editing the aup file to fix the aliasfile locations? Is there a way for these to be relative paths?

I would highly recommend that you “resolve all dependencies” (copy all dependencies into the project) before moving the project.
I would highly recommend that you do not manually edit the AUP file or the _data folder in any way. Always use Audacity for modifying projects. It is extremely easy to turn hours of work into trash by manually editing any part of the project structure.
See here for how to safely move a project: Audacity Manual

  • Is there anything else about On Demand Loading that I need to know? I understand that I have to be careful not to move the wav files, and I also assume any edits I do will be performed on the original files.

The original files are never changed in any way. Audacity does not work on files at all. Audacity copies audio data from the files and works with the copied data. What you are doing with “On Demand” is deferring the copying of the audio data until it needs to be copied.
Audacity can accomplish simple “cut/copy/delete/trim” type edits without copying data, Simple edits are just a matter of juggling the “block files” into the correct order and taking care of the “end bits” (an edit point is unlikely to line up exactly with a block boundary).
“Processing” tasks (where the audio sample values are changed) requires that Audacity has a copy of the audio data to work with. So, for example, if you import a large file with “On Demand” loading and then amplify the entire track, Audacity will “load” the file without copying the data, but to create the amplified version it will copy the data (from the original file), amplify it a block at a time, and write the amplified data into the _data folder. If you only amplify part of the track, then only the part that is being processed is copied. “On Demand” loading means that Audacity only copies the audio data from the file when it needs to.