Audacity getting confused about compressed projects?

This read-only archive contains discussions from the Adding Feature forum.
New feature request may be posted to the Adding Feature forum.
Technical support is available via the Help forum.
Tid
Posts: 24
Joined: Mon Aug 04, 2014 12:09 pm
Operating System: Please select

Re: Allowing 'Compressed only' projects

Post by Tid » Tue Aug 05, 2014 10:25 am

Gale Andrews wrote:You are very welcome to vote for any of:
  1. Opening files directly like a media player, perhaps with a waveform, but no editing possible.
  2. Reopened compressed projects store their PCM data in the Audacity temp folder. Only OGG files are in the _data folder for the compressed project. When you save the project, the OGG files are overwritten. When you close the project, the PCM data is deleted. Reopening that project repeats the process on first re-open, reading the OGG files and writing the PCM data for the imported OGG files to the temp folder.
  3. Reopened compressed projects store OGG files plus their PCM data in the _data folder for the compressed project, as now, but unused PCM data that was for Undo/Redo is deleted when you save the project (or perhaps when you close it). Reopening the project reads not from the OGG files but the PCM data.
1) probably won't happen .

3) should probably be done.

2) is probably more work than 3) but is closest to what you are asking for. I think at the moment 2) makes most sense. Otherwise you can't send identically named iterations of the same compressed project to others - to send version 2 of the compressed project to someone else containing only an OGG, you have to save it as a new compressed project.

Differently named iterations are a good practice for safety, but not so convenient.

The choice between 2) and 3) could be a preference, but probably not needed.

Gale
As a person who uses Audacity for possibly unusual reasons, this might be the point I should explain my uses of it in greater depth.

Originally, I used Audacity very simply, to import analogue vinyl and cassette tracks, remove any extraneous stuff at the start / end, and export the song as an AAC to iTunes, where I could apply equalising to my own taste. Then close the project without saving changes, as its work had been done.

More recently I have begun to use the software to create evidence in an anti-piracy case : there are supposed cover ('tribute') versions of songs available for download purchases online. However, the people responsible have used original recordings in some cases, rather than the cover versions they are claimed to be. I am compiling proof of this by importing the original track (Beatles, Stones, Led Zeppelin, Genesis, Bob Dylan, The Smiths, The Beach Boys ... yes, really! among others), then the supposed cover version. I synchronise the two waveforms, then play them. Apart from a bit of 'phasing', they are quite clearly a single recording and the waveforms - apart from amplitude or tonal variations - are identical. (Very close 'genuine' cover versions are nevertheless obviously different in Audacity - the tempo varies slightly, which accumulates over the course of the song, and the vocals are never the same. The waveforms are similar, but not identical.)
To save all these projects would incur file sizes of around 100MB (or more) per song, which is excessive as each project consists of 2 imported MP3s no larger than around 5MB each. Discovering the 'Compressed Copy' option was a godsend to me, as it meant I could do 2 things : 1) export an AAC copy of the two tracks mixed down to one, demonstrating that in each case, the original recording had been used, and 2) keep the waveforms as extra evidence.
What completely threw me was that though I could achieve 1) and 2) and keep file sizes down, if I wanted to look at the waveforms again, suddenly there would be a large folder of PCM files, defeating the object of what I was using the Compressed Copy option for to begin with. I don't mind that Audacity must have the PCM data to work with, but it just seems common sense to me that it would be deleted again for Compressed Copy projects, and recreated anew (then deleted again) each time the project was reopened.

So my "vote" is as follows :

Not for 1., which I've never said I wanted - the fact that Audacity appears to play the Ogg Vorbis tracks but in fact doesn't, is unimportant to me. The tracks play, that's all that matters; it's just a matter of what happens to the PCM data after one closes the project.

Yes, I'd vote for 2. as it seems the commonsense solution for people who only want to work with a compressed project.

Not for 3., as storing the PCM data in the project _data folder defeats the whole purpose of having a Compressed Copy option to start with - surely people only use that option to reduce the size of projects? Storing a folder of PCM data AS WELL for each project, makes no sense that I can see.

Having a preference to choose between 2. and 3. WOULD actually be useful, but only if you think 3. is a workable option in the light of what I've said above.

(There is one further thing I don't know the answer to : you've talked a lot about the use of PCM data in Audacity's temp folder : does this temp folder get deleted when a project is closed? Or - if one has several projects - does it only hold the data for the project currently open? Can I manually delete that data if I can tack it down? Would that affect Audacity's ability to read Ogg Vorbis files, create a temporary set of PCM files, then delete them again?)

steve
Site Admin
Posts: 81609
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: Allowing 'Compressed only' projects

Post by steve » Tue Aug 05, 2014 10:47 am

Tid wrote:does this temp folder get deleted when a project is closed?
Yes, unused temporary data is deleted, provided that the project is closed correctly (not just switching off your computer, which you should never do while a project is open). In the case of a "Compressed Copy", the audio data extracted from the OGG file(s) is not "unused" but an essential part of the "un-packed" project.
Tid wrote:Can I manually delete that data if I can tack it down?
No, the temp data should never be manually deleted or modified in any way. Doing so is very likely to destroy the entire project. See here for how to safely manage Audacity projects: http://manual.audacityteam.org/o/man/au ... jects.html

You can however delete the "e00" folder IF you saved a "Compressed Copy" back into the original _data folder, but be careful doing that. If you do not explicitly save a "Compressed Copy", the the data in the "e00" folder is required by the project, and deleting it will totally destroy the project. (see here in the manual: http://manual.audacityteam.org/o/man/fi ... compressed). It is MUCH safer to just make a copy of the project (as I described in my next post) if you just want to "view" the project without changing anything.
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)

steve
Site Admin
Posts: 81609
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: Audacity getting confused about compressed projects?

Post by steve » Tue Aug 05, 2014 11:12 am

Tid wrote:Audacity 2.0.3
Mac OS X 10.6.8
Audacity 2.0.3 has been superseded. At your earliest convenience you should update to the current version which is available here: http://audacityteam.org/download/mac
Tid wrote:I've only recently discovered the option to Save Project As A Compressed Version.... I choose this because the imported audio is MP3, and each project is a mix of two MP3s, so a compressed Ogg Vorbis version is far preferable to having a full project which may be 150MB.
Please be aware that each time the audio data is compressed it causes additional damage to the sound quality. That damage is unavoidable and irreparable. Compressed formats were developed to aid delivery where transporting large files is impractical. In these days of cheap data storage and broadband, there should be little need to use compressed formats other than for compatibility with portable media players. Where sound quality is important, compressed formats should not be used during the production process.
Tid wrote:The problem is when I open that project again. Whether I make any changes, or simply play the project, Audacity seems to ignore the fact that it's just opened a Compressed project (though I can see it physically open the 2 Ogg Vorbis tracks) - when I close it and go to the project data folder, I see that as well as the 2 Ogg Vorbis files, there is now a subfolder of all the many small full size Audacity "pieces", in other words the very data I tried to avoid when I first created the project.

How do I 'persuade' Audacity not to do this, in other words to open a compressed project and keep it compressed? :-
As Gale has explained, Audacity only works with uncompressed audio data. That is an important part of its design and enables Audacity to perform the range of editing and processing that it does.

The simple solution is to copy the AUP file and the associated _data folder to a new folder. For example, if you have a project called "test", then you will have a file called "test.aup" and a folder called "test_data". Make a new folder and give it a name, for example "copy". Then copy the "test.aup" and "test_data" folder into "copy". You can now open the project in the "copy" folder without affecting the original. If you don't want to keep it you can delete the entire "copy" folder when you have finished.

Tid wrote:What completely threw me was that though I could achieve 1) and 2) and keep file sizes down, if I wanted to look at the waveforms again, suddenly there would be a large folder of PCM files, defeating the object of what I was using the Compressed Copy option for to begin with.
Perhaps so, but entirely consistent with what the "Compressed Copy" feature is designed and intended for. which is to provide an editable copy of the project that can be easily transported (by e-mail or on disk or similar) when sound quality is not of great importance.

Audacity allows you to do all sorts of things that it was not intended / designed for, but when using Audacity (or anything else) in ways that were not intended, it is reasonable to expect that you may need to "workaround" some design features. In this case, copying a project should not take more than a few seconds of your time.
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)

steve
Site Admin
Posts: 81609
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: Allowing 'Compressed only' projects

Post by steve » Tue Aug 05, 2014 11:34 am

Gale Andrews wrote:2. Reopened compressed projects store their PCM data in the Audacity temp folder. Only OGG files are in the _data folder for the compressed project. When you save the project, the OGG files are overwritten. When you close the project, the PCM data is deleted. Reopening that project repeats the process on first re-open, reading the OGG files and writing the PCM data for the imported OGG files to the temp folder.
The problem that I see with that is that it would appear that the original OGG file had been retained, when in fact it has been replaced by a lower quality copy (due to lossy compression). Consequently I think that it would be a necessary default to show a warning, which would be a minor inconvenience to people that use Compressed Copies in the way they were intended to be used.
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)

Tid
Posts: 24
Joined: Mon Aug 04, 2014 12:09 pm
Operating System: Please select

Re: Allowing 'Compressed only' projects

Post by Tid » Tue Aug 05, 2014 2:05 pm

steve wrote: The problem that I see with that is that it would appear that the original OGG file had been retained, when in fact it has been replaced by a lower quality copy (due to lossy compression). Consequently I think that it would be a necessary default to show a warning, which would be a minor inconvenience to people that use Compressed Copies in the way they were intended to be used.
Steve, thanks to both you and Gale for all your help. Part of my problem has been a failure to understand some of the core fundamentals of Audacity (partly caused by its not being a Mac program!! LOL). I've assumed all along that the Compressed Copy option was a way to keep project sizes down when using audio that started life as a lossy format MP3 or AAC.

However, I do understand about lossy formats both in images and audio. I do understand that 'lossy' means that part of information is lost when you compress, and can never be recovered. What a lot of people ignore though, is that noticeable degradation only occurs when you compress from one level of quality to a lower one (e.g. from 256kbps MP3 to 128kbps.) There is a TINY degradation when you import a 128, convert to uncompressed, then rewrite as 128 again - but you would have to convert many times before your ear picked up the difference in quality (the same applies when you edit and rewrite a JPEG to the same particular compression 'quality' setting : it's quite a few conversions before you notice anything). If I choose to edit Ogg Vorbis files a few times in Audacity, not only have I freely chosen and accepted a level of degradation, it's not going to show up in a few edits as each time Audacity applies the same compression algorithm to get an Ogg Vorbis file, as it did before. Once you've 'thrown away' certain frequencies, you can't throw them away again simply because they're not there to throw away.
Last edited by Tid on Tue Aug 05, 2014 2:13 pm, edited 1 time in total.

Tid
Posts: 24
Joined: Mon Aug 04, 2014 12:09 pm
Operating System: Please select

Re: Audacity getting confused about compressed projects?

Post by Tid » Tue Aug 05, 2014 2:11 pm

steve wrote: Please be aware that each time the audio data is compressed it causes additional damage to the sound quality. That damage is unavoidable and irreparable.
See my previous reply.
Tid wrote:The problem is when I open that project again. Whether I make any changes, or simply play the project, Audacity seems to ignore the fact that it's just opened a Compressed project (though I can see it physically open the 2 Ogg Vorbis tracks) - when I close it and go to the project data folder, I see that as well as the 2 Ogg Vorbis files, there is now a subfolder of all the many small full size Audacity "pieces", in other words the very data I tried to avoid when I first created the project.
steve wrote: As Gale has explained, Audacity only works with uncompressed audio data. That is an important part of its design and enables Audacity to perform the range of editing and processing that it does.

The simple solution is to copy the AUP file and the associated _data folder to a new folder. For example, if you have a project called "test", then you will have a file called "test.aup" and a folder called "test_data". Make a new folder and give it a name, for example "copy". Then copy the "test.aup" and "test_data" folder into "copy". You can now open the project in the "copy" folder without affecting the original. If you don't want to keep it you can delete the entire "copy" folder when you have finished.
Thanks for that suggestion - I'll give it a try, and use the "copy folder" each time I open one of my piracy projects (there's between 15 and 20 so far, each converted to Ogg Vorbis).

Tid
Posts: 24
Joined: Mon Aug 04, 2014 12:09 pm
Operating System: Please select

Re: Allowing 'Compressed only' projects

Post by Tid » Tue Aug 05, 2014 2:25 pm

Yes, that works fine. :D I'll keep the copy folder, but just trash its contents after each time I use it.

Just one further question -
  • I make a copy, but this time I do an edit.
    I save the edit back to the copy folder as a Compressed Copy
    I move the .aup and Ogg Vorbis files back to their original folder but trash the PCM folder


If I then copied those files to the copy folder in the future, would double clicking the .aup work, or would I get an error?

No need to reply - I just tested and that works too :)

Thanks for all your help. :) :)

steve
Site Admin
Posts: 81609
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: Allowing 'Compressed only' projects

Post by steve » Tue Aug 05, 2014 3:45 pm

Tid wrote:Once you've 'thrown away' certain frequencies, you can't throw them away again simply because they're not there to throw away.
Unfortunately it's not quite like that. The losses may be minimised by encoding at a higher bit rate (larger file size) than the original encoding, but "lossy" encoding always reduces the sound quality each time. In effect, it finds some new part to throw away each time.

Compare these two files. One is a first generation Ogg file with quality setting 5. The other is a 7th generation file with quality setting 5 (each time).
Attachments
1st_generation.ogg
(370.19 KiB) Downloaded 53 times
7th_generation.ogg
(352.61 KiB) Downloaded 54 times
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)

Gale Andrews
Quality Assurance
Posts: 41761
Joined: Fri Jul 27, 2007 12:02 am
Operating System: Windows 10

Re: Allowing 'Compressed only' projects

Post by Gale Andrews » Tue Aug 05, 2014 4:08 pm

Tid wrote:Yes, that works fine. :D I'll keep the copy folder, but just trash its contents after each time I use it.

Just one further question -
  • I make a copy, but this time I do an edit.
    I save the edit back to the copy folder as a Compressed Copy
    I move the .aup and Ogg Vorbis files back to their original folder but trash the PCM folder


If I then copied those files to the copy folder in the future, would double clicking the .aup work, or would I get an error?

No need to reply - I just tested and that works too :)
So the objective is to have two versions of each song project you which you can easily open to see the waveforms: 1) containing the two comparison tracks 2) containing the single track mixed down from both? And 1) is a compressed copy of a project?

I think that will work well, but you should close projects before copying them manually or Audacity may complain about orphan block files (files for the Undo/Redo mechanism) when you reopen the copied project.

But you will still get the PCM data in the compressed project once you open it.


Gale
________________________________________FOR INSTANT HELP: (Click on Link below)
* * * * * Tips * * * * * Tutorials * * * * * Quick Start Guide * * * * * Audacity Manual

Gale Andrews
Quality Assurance
Posts: 41761
Joined: Fri Jul 27, 2007 12:02 am
Operating System: Windows 10

Re: Allowing 'Compressed only' projects

Post by Gale Andrews » Tue Aug 05, 2014 4:42 pm

In http://forum.audacityteam.org/viewtopic ... 17#p250817 I meant to say that the idea you quote below 2) seemed to make sense to me (I corrected that post).

3) (not tidying up undo/redo files after closing an edited compressed project) is really a bug as I see it.
steve wrote:
Gale Andrews wrote:2. Reopened compressed projects store their PCM data in the Audacity temp folder. Only OGG files are in the _data folder for the compressed project. When you save the project, the OGG files are overwritten. When you close the project, the PCM data is deleted. Reopening that project repeats the process on first re-open, reading the OGG files and writing the PCM data for the imported OGG files to the temp folder.
The problem that I see with that is that it would appear that the original OGG file had been retained, when in fact it has been replaced by a lower quality copy (due to lossy compression). Consequently I think that it would be a necessary default to show a warning, which would be a minor inconvenience to people that use Compressed Copies in the way they were intended to be used.
The way they were intended is quite unintuitive IMO (you reopen a compressed project, but it no longer is compressed). :? This isn't the first time users have reopened a compressed project and deleted the _data folder, thinking it superfluous whereas it's the OGG that's superfluous. Moreover the OGG doesn't change if you edit and resave the compressed project - drag it into Audacity after resave and it's the same as the OGG you started with :?

Also I think the inconvenience of not being able to save iterations of the same compressed project is quite a severe limitation of this feature.

The fact that there would be no _data folder in the compressed project should make it intuitive, but the warning when you save the compressed project could be adjusted to the new behaviour, or there could be a choice offered between the new or the (fixed) current behaviour. I am not convinced the choice is needed, because if the user wants a lossless copy of their edited and resaved compressed project they can do so by File > Save Project As...

If we went for behaviour 2) then I think it would be desirable to give compressed projects a separate extension to distinguish them from AUP.


Gale
________________________________________FOR INSTANT HELP: (Click on Link below)
* * * * * Tips * * * * * Tutorials * * * * * Quick Start Guide * * * * * Audacity Manual

Locked