What happened to 64 bit integers in Audacity?

[ScottyPcGuy_03]

Hey guys,

From the wiki:

In principle, the solution to recording over 13.5 hours is to use the latest 1.3 Beta version of Audacity. Here, samples are stored as 64-bit values (even on 32-bit machines), which at 44100 Hz sample rate offers a maximum recording length of a mere 58 billion hours. Whilst we believed 64 bit storage worked properly in 1.3.3 and later, some reports in early 2008 suggested that recordings were still coming up against the 2^31 limit. Our development code was fully converted to 64 bit sample counts in July 2008, so long recordings should work again in our Nightly Builds and in 1.3.6 Beta onwards.

So why doesn't my 42GB project open without complaining about orphaned blockfiles, even when I edit a copy of the project to include one track? (it only displays 6.5 hours, but I recorded 7.5hrs @ 96kHz)

I'm using Audacity 1.3.12,
Win7 x64,
2.4GHz Core 2 Duo,
2GB Ram


Thanks,
Scott

[kozikowski]

Large shows are an exercise in computer technology. It's not just "the OS handles it so everything must be OK." Everything is not OK. If the show goes much over about six hours, the computer can't be doing anything else. There is one developer that got many multiple hours of capture to work but it wasn't pretty and he didn't use a conventional office PC.

You probably shouldn't be connected to a network and you shouldn't have virus software running. Does your machine survive the Microsoft Crash Analysis -- multiple times?

http://oca.microsoft.com/en/windiag.asp

Then you need tons of free disk space thoroughly error checked and defragmented. You need way more space than you think by just adding up all the required show space. If Windows itself chokes for any reason, you're toast.

There are other restrictions that I can't remember because I don't use fancy Windows tools on a regular basis.

The other thing we tell people trying to do this is you don't want Audacity. Over six hours you want a surveillance program.

Koz

[steve]

So why doesn't my 42GB project open

So you managed to create a 42GB project - that would suggest that it is "in principal" working. Unfortunately, the fact that the project won't open could be due to almost anything. Handling very large projects is notoriously difficult, and you are really pushing the capabilities of your hardware. Audacity itself has pretty modest hardware requirements, but the minimum specification for running Windows 7 64 bit is 1GHz processor and 2 GB RAM. Microsoft usually recommend using hardware that is at least double the minimum specification, though in the case of Windows 7 they appear to be rather shy about advertising a recommended specification.

I'm curious why you have such a huge file recorded at such high resolution.

[Gale Andrews]

If you have a 42 GB project file, clearly that's more than the data you recorded, which would be about 10 GB for a mono 96000 Hz track of 7.5 hours' length in a 32-bit project (about 20 GB for stereo).

It would definitely seem that Audacity can handle more samples then 2 147 483648 (2^31) outside saved projects. On Windows 7 in 1.3.13 alpha, I generated a 4 hours' mono tone at 192000 Hz, which should give me 2 764 800000 samples. The track can be successfully played anywhere, and edits made in the last 5 minutes of the track where the timeline is at more than 2 ^ 31 samples. This is despite Selection Toolbar being unable to display the "1" or "2" for one or two thousand million samples, and displaying garbage above 2 147 483648 samples.

It does look though like it can't handle projects with more than 2 ^ 31 samples. If I generate and immediately save a tone as above, then re-open the project, I get 10547 orphaned blockfiles (as many as the project contains). If I either continue without deleting the orphans or do delete them, the track remains empty, though Audacity remains responsive. So it looks as if there is something to investigate.

Bear in mind also that few players will play WAV or AIFF larger than 2 GB in size, which means that even if you exported as a 44100 Hz 16-bit mono WAV, you're limited to 6 hours 40 minutes.



Gale

[kozikowski]

<<<Bear in mind also that few players will play WAV or AIFF larger than 2 GB in size, which means that even if you exported as a 44100 Hz 16-bit mono WAV, you're limited to 6 hours 40 minutes. >>>

I thought it was 4GB, but I believe you.

What's the upper limit for MP3?

Koz

[Gale Andrews]

<<<Bear in mind also that few players will play WAV or AIFF larger than 2 GB in size, which means that even if you exported as a 44100 Hz 16-bit mono WAV, you're limited to 6 hours 40 minutes. >>>

I thought it was 4GB, but I believe you.

The WAV/AIFF limit should indeed be 4 GB (the size can't go higher because those formats use a 32-bit unsigned integer to record the file size header). The problem is that many programs treat the file size header as a signed integer, and that makes the limit 2 GB.

Also the FAT32 file system has a file size limit of 4 GB. So even if a program supports RF64 WAV standard which has a 64-bit header and so removes the 4 GB restriction, people on FAT32 won't be able to use it.

What's the upper limit for MP3?

"AFAIK" there isn't a practical limit. FLAC used to be limited to 2 GB but now isn't, except there seem to be reports that builds of FLAC for Windows still have the 2 GB limit because of Microsoft limitations.



Gale

[ScottyPcGuy_03]

Thanks for the feedback guys... Sorry I took so long to get back to y'all; school and work have been getting crazy.

Anyway,

@Steve:
I recorded my Church's Festival of the Arts in 6 channel audio (drums, piano, bass, choir, etc.) for later mixdown to stereo. I failed to mention that I wasn't thinking and so I forgot to reset the project rate to 44.1kHz Had I done so it probably would have worked.

@Gale:
I forgot to mention that I recorded SIX channels @ 96kHz, 24bit. (not two or at 32bit ) I also failed to mention that I am going to discard much of the audio as the Festival I was recording was actually only 2hrs long and repeated three times that day; I wanted multiple takes. The extra hour and a half I obtained by starting the recording early to ensure Audacity was going to keep recording for that long. (I had trouble keeping Cubase going, which is why I tried Audacity.) Had I remembered the 32bit limit I would have stopped and recorded more audio in another project.

I've decided that since I probably won't be able to open my project file until audacity goes fully 64-bit, (and I can't wait that long) I'll just write a program that splits up my project file into multiple projects of a more reasonable size, separated by silence whenever possible. If that's not possible, please let me know.

Regards,
Scott

[kozikowski]

<<<I recorded my Church's Festival of the Arts in 6 channel audio (drums, piano, bass, choir, etc.) for later mixdown to stereo. I failed to mention that I wasn't thinking and so I forgot to reset the project rate to 44.1kHz Had I done so it probably would have worked. >>>

You stepped in one there. Multiple people would kill to know how you got six high quality channels into Audacity. Black Arts? Wiccan Teachings? Garlic?

Koz

[Gale Andrews]

...I've decided that since I probably won't be able to open my project file until audacity goes fully 64-bit, (and I can't wait that long) I'll just write a program that splits up my project file into multiple projects of a more reasonable size, separated by silence whenever possible. If that's not possible, please let me know.

The main problem is that we don't know why you have those orphaned blockfiles. I suppose you could theoretically cut individual blocks of lines out of an .aup file representing wavetracks or parts thereof and paste to a new aup file with the appropriate headers, but you could still get file consistency errors.

The usual recommendation would be to group the .au files in the _data folder into folders of about 1 GB each, sort the .au files into time modified order, then rename them into a consecutive alphanumerical sequence. Then you can in theory use the "Crash recovery utility" to make (on a per folder basis) one WAV file for each channel that you had. Details on the Crash Recovery page.

I suppose depending on how much you can open and actually plays, you could export those parts as WAVs (maximum 2 GB), then try and recover only the last hour. If that is what is missing, those .au files will be the later ones which you can check against the file timestamp.




Gale