Audacity Forum

otwo_pipes wrote:PGA wrote:

Sample rate x number of seconds x sample format x number of channels = total number of bits
Total number of bits / 8 = total number of bytes
Total number of bytes /1024 = total number of KBytes
Total number of KBytes / 1024 = total number of MBytes

So, if recording CD quality (Sample rate=44100, sample format = 16bit), the actual numbers for one minute become:
44100 x 60 x 16 x 2 / 8 / 1024 /1024 = 10.094MB (to 3 decimal places)

and all I can say is I whole heartedly agree.

The formula is the (first) one I fished from the web - not from Audacity documentation. You're right - it's a shortcut that is "roughly" correct (but the result a little too large) because it uses an implied divisor of /1000 (to take kHz input) rather than a second divisor of /1024.

But I think we can agree that PGA's formula or the one at http://tinyurl.com/cb7tc3k will produce the result of 0.549 MB per second for 24x96 stereo capture, *when Audacity writes to disk*, correct? You can verify this by turning off audio cache and looking in the temporary folder.

otwo_pipes wrote:

Gale Andrews wrote:However you play with formulas, you can concur I think that 24x96 recording requires some 33 MB per minute of RAM or disk space just by looking at one minute of
recording (with audio cache disabled) in the Audacity _data folder.

this is NOT what I see. As I said in my last posting, I have started to use Microsoft's' Process Monitor to look at memory usage and I have no reason to doubt the results obtained from the use of this program. My results are (with audio cache disabled);-

For 1 minute of recording, averaged from 6 measurements, recording at 16/96 the memory used was 23.2 MB/minute
For 1 minute of recording, averaged from 6 measurements, recording at 32/96 the memory used was 45.8 MB/minute
and the real interesting one is of course 24/96 capture whereby we have
For 1 minute of recording, averaged from 6 measurements, recording at 24/96 the memory used was 45.8 MB/minute

My testing to obtain these results was identical and my interpretation of these measurements is Audacity is using 4 bytes per sample for both 24 and 32 bit recording.

Yes, I believe this is what Audacity is doing when recording to RAM in 24-bit, for obscure reasons. See http://bugzilla.audacityteam.org/show_bug.cgi?id=50 .

However I don't think that is anything to do with the Audio Cache problems. What I am finding on Windows 7 (consistently over numerous tests and irrespective of 24-bit RAM recording taking RAM amounts corresponding to 32-bit disk capture ) is that about 6 MB more RAM is being used than would be expected for disk capture. For one minute of stereo audio:

* 44100 Hz x 16-bit is about 16 MB taken in RAM instead of 10 MB on disk
* 44100 Hz x 32-bit is about 26 MB instead of 20 MB on disk
* 96000 Hz x 16-bit is about 29 MB instead of 23 MB on disk
* 96000 Hz x 32-bit is about 52 MB instead of 46 MB on disk

So that third and fourth items (96000 Hz) contrasts with what you are seeing by taking more RAM than expected. There is no swapping to page file during recording. This is on a machine that does not crash with audio cache enabled, just using Windows Task Manager. This "could" be a red herring but is more than a "formula" difference.

If you do audio cache "on versus off" comparisons for those four settings, do you see anything like that?

otwo_pipes wrote:I wonder if:-

Each .au file is 1 MB in size by default.
This is not what I seeing, my files are 780MB and not, as you say, 1MB. I don't know the relevance of this but I thought it was worth mentioning.

I agree the block file size for 24-bit (irrespective of Audio Cache being on or not) is 780 kb. I am not sure why that is at the moment. The block file size for 32-bit and 16-bit is 1 MB.

has anything to do with the file size for 24 and 32 bit recording being identical to all intents and purposes.

Seemingly not directly, because you see the same block size when writing to disk.

otwo_pipes wrote:I can see 1GB of RAM has been used because I start with 1.3GB and have 300MB free when the system crashed but I do not see how you can say 300MB has been swapped to page file.

That was assuming you had used 1.3 GB of RAM but it is now apparent you would be using more RAM, because 24-bit recording to RAM uses space equivalent to 32-bit on disk (about 46 MB per minute for you, 52 MB for me for 24x96 stereo). But when recording stops and the cached audio is saved to disk, the size goes back to the expected 33 MB.

otwo_pipes wrote:

Also I note in your last image above of the folder structure, there was no d0c folder?

Correct:

And had it already populated d00 - d99?

No. Look again at the main Windows Explorer image and the sub directories of the e00 directory. The directories are d0a -d0e with d0c missing and d06 - d09 only. There are no directories d00 - d99.

I don't understand why the sequence does not go d00, d01, d02, d03... as I think is intended. It should not get to d0c before having got past d99.

otwo_pipes wrote:I do not know if this is relevant or not but when Audacity crashes it leaves, quite obviously, many directories and sub directories in the project directory. After a crash I always restart my system so I have a reference point to continue from. Audacity does not always clean the project directories correctly when re-starting i.e I have seen 4 project directories, one in use and 3 from previous crashes, when I took the directory screen shots. When Audacity crashes and I re-start Audacity asks if I wish to recover the data.

If Audacity wrote to disk and populated the temp folders, then it should never clean them when restarting, because crash recovery depends on them.

otwo_pipes wrote:The last testing I performed on Friday evening did NOT produce a crash on my Win7 system no matter what sample rate, bit depth or time tests were performed... The only change to procedure I started to adopt was I have deleted the Audacity temp directory due to Audacity not cleaning directories correctly when re-starting after a crash.

This could be relevant. Please start with empty temp directory and if no crashes, force quit to leave the temp data full, recover, then record.

That is if this isn't already driving you spare, of course. Thanks for all the tests.

There is some chance we will remove this feature for 2.0.2 or increase the default minimum available memory. We will take a look at possible choices.



Gale

Gale Andrews wrote:I don't understand why the sequence does not go d00, d01, d02, d03... as I think is intended. It should not get to d0c before having got past d99.

Gale,
You are assuming the digits are decimal digits. But if the letter "c" is a valid digit, these must be hexadecimal digits. Therefore the sequence would be: 01, 02, ... , 09, 0a, 0b, 0c, 0d, 0e, of, 10, 11, ... , 19, 1a, 1b, ... etc.

But I think we can agree that PGA's formula or the one at http://tinyurl.com/cb7tc3k will produce the result of 0.549 MB per second for 24x96 stereo capture, *when Audacity writes to disk*, correct? You can verify this by turning off audio cache and looking in the temporary folder.
Correct:

When I wrote:-

this is NOT what I see. As I said in my last posting, I have started to use Microsoft's' Process Monitor to look at memory usage and I have no reason to doubt the results obtained from the use of this program. My results are (with audio cache disabled);-

I should have said the audio cache was enabled and not disabled. I was measuring RAM usage not disk usage and therefore the cache has to be enabled. Your question which follows caused me to look again at my test system to see how I had left the settings. Apologies for the wrong information and I hope this did not cause any wasted time.

Gale wrote:

If you do audio cache "on versus off" comparisons for those four settings, do you see anything like that?

How am I to measure disk usage for the four measurements? Do I use explorer and look at the memory used in the sub directories beneath the e00 directory?
How do I do the measurements with audio cache on. Do I have to wait until the cache is full and the audio is written to disk or just stop the recording part of the way through thereby instigating the disk write?
If you want me to produce results from my measurements maybe it is worthwhile me taking the measurements in the same manner you have made your measurements.

This could be relevant. Please start with empty temp directory and if no crashes, force quit to leave the temp data full, recover, then record.

How do I force quit? I ask so I don't waste time performing the wrong testing.

That is if this isn't already driving you spare, of course. Thanks for all the tests.

When I get my teeth into something I do not quit. However, in this case I may not have the tools in order to find a solution but time and effort is not a problem.

There is some chance we will remove this feature for 2.0.2 or increase the default minimum available memory. We will take a look at possible choices.

You may not like this input: - by all means remove the feature as it seems to cause more trouble than the worth but I strongly suggest the root cause of the crash is found even if the feature is removed. My experience is not resolving the root cause will only lead to greater problems at a later date. As I said, you may not like my input but.........

PGA wrote:You are assuming the digits are decimal digits. But if the letter "c" is a valid digit, these must be hexadecimal digits.

The normal sequence for the project data starts at d00 (which is the salient point being discussed).
The numbering is indeed hexadecimal, so will (probably, I've not tested a big enough recording) go up to "dff".
As Gale wrote, "It should not get to d0c before having got past d99."

The sequence should go,
d00, d01, d02, d03,...d09, d0a, d0b, d0c, d0d, d0e, d0f, d10, d11...

otwo_pipes wrote:When I wrote:-

this is NOT what I see. As I said in my last posting, I have started to use Microsoft's' Process Monitor to look at memory usage and I have no reason to doubt the results obtained from the use of this program. My results are (with audio cache disabled);-

I should have said the audio cache was enabled and not disabled.

No problem, I knew what you meant.

otwo_pipes wrote:

If you do audio cache "on versus off" comparisons for those four settings, do you see anything like that?

How am I to measure disk usage for the four measurements? Do I use explorer and look at the memory used in the sub directories beneath the e00 directory?
How do I do the measurements with audio cache on. Do I have to wait until the cache is full and the audio is written to disk or just stop the recording part of the way through thereby instigating the disk write?
If you want me to produce results from my measurements maybe it is worthwhile me taking the measurements in the same manner you have made your measurements.

For RAM usage, I was just looking in Windows Task Manager (Performance tab) at the "Available" figure under "Physical Memory (MB)".

I suggest you only record for one minute or possibly two or three minutes to RAM then press Stop. This will mean you almost certainly won't get any swapping to page file which would increase the "available" figure and confuse the issue.

Since I have 6 GB RAM I did record for quite long. So for example I got about 165 MB of RAM used for 10 minutes of 16 x 44.1 stereo recording whereas if it was disk write it would only use a little over 100 MB.

For disk write, similarly press Stop after one minute and look in Explorer at your Audacity temp folder for the number of .au files and the space each takes. You can see the location of your Audacity temp folder in the "Directories" Preferences (above "Audio Cache"). But 16 x 44.1 MB in stereo is about 10 MB of disk write as proved by the "formulas", so you hardly need to look.

otwo_pipes wrote:

This could be relevant. Please start with empty temp directory and if no crashes, force quit to leave the temp data full, recover, then record.

How do I force quit? I ask so I don't waste time performing the wrong testing.

The same way you would force quit any program, using Windows Task Manager. The best way to make sure you force quit properly is to click the "Processes" tab in Task Manager, right-click "audacity.exe", choose "End Process Tree" then confirm.

Or another way, turn Audio Cache off, File > Close to clear the temp directory, then Generate > Tone... and generate say 20 minutes of tone. This populates the temp directory. Now turn Audio Cache on and record to RAM.

otwo_pipes wrote:

There is some chance we will remove this feature for 2.0.2 or increase the default minimum available memory. We will take a look at possible choices.

You may not like this input: - by all means remove the feature as it seems to cause more trouble than the worth but I strongly suggest the root cause of the crash is found even if the feature is removed. My experience is not resolving the root cause will only lead to greater problems at a later date. As I said, you may not like my input but.........

If we remove the feature it will be wrapped in an "experimental" define and the define commented out in src/Experimental.h. This means that Audio Cache will not be in releases but anyone building Audacity will be able to uncomment the define and so make the Audio Cache checkbox appear in Directories Preferences. This will allow us to work on correcting the issues in this feature.

However in this case I very much doubt the issues in Audio Cache are causing other issues when the feature is turned off. We can prove that by releasing a build with Audio Cache made experimental and so unavailable.



Gale

The good news is my Win7 system is back to repeatable crashes. I have not made any changes or installed any sw. Sorry but I cannot give any pointers as to what may be happening. Almost no problems have been experienced with Audacity during LP ripping with the audio cache disabled apart from one day when there were many occasions when Audacity recording could not be started necessitating a PC re-boot. More on this later as the same scenario has been seen after a crash.
I started the current tests after I had finished the days LP ripping session.
For testing, the audio source was digitised using my external ADC box set to 24/96. The digitised audio was connected to the M-Audio sound card via S/PDIF connections. The function of the M-Audio card is solely to provide a S/PDIF - S/PDIF interface. The Audacity audio cache was enabled and set to 16MB with a data capture rate of 24/192.

An Audacity recording session was started with the settings described. Data was captured for 19:22 when there was a crash with an 'Exception,' message. A screen shot of the 'Exception' message is available if required. I have seen Audacity crash with an 'Exception' error once before. The other crashes are identified by the scrolling waveforms freezing. After the crash I carried out the following sequence:-
Select Re-try from the 'Exception' message to close Audacity
Start Audacity as normal
Do not recover project when Audacity prompts for the required action to continue
Start recording: however, recording does not start. Normally, when I start recording, the play and record meters are active, the audio waveform capture screen is running and the time stamp and vertical cursor are scrolling. The only sign of recording trying to start is the vertical cursor blinking but not moving. The only way out of this scenario is to close Audacity and re-boot the PC. I have also seen this fault, on numerous occasions, when recording with the audio cache disabled.
(I suspect deleting the Audacity temp directory, after closing Audacity may also work but one thing at a time please.)
Start taking notes regarding results from the previous and the following tests.
Close Audacity
Re-boot
Start Audacity as normal
Start recording
This recording crashed at 19:09
Repeat above procedure and Audacity crashed at 19:24

I was monitoring the directory writes on the second crash and I suspect Audacity crashed when trying to write a .au data file to disc. I was watching the .au files being created and it seemed to be the correct timing for the next .au file to be created. This is only a gut feel and no times were measured however this may be a useful pointer. I was monitoring the available memory at the time of the third crash and this was oscillating around the 85-105 MB mark even though the audio cache was set to 16MB.

The data results I have gleaned from the three crashes is as follows:-
Directories were correctly created in sequence of d00 - d06
The 1st crash had 18 .au files in the d00 directory
The 2nd crash had 20 .au files in the d00 directory: all files were 780kB apart from 2 files which were 318kB. (Why are there two files of size 318kB when Audacity crashes? I could understand all files being 780kB and one file was a different size but two files of 318kB, seems odd to me.)
The 3rd crash had no .au data files in any directory.

Let me know your views ideas and document any tests you require me to perform.

Gale wrote:

If we remove the feature it will be wrapped in an "experimental" define and the define commented out in src/Experimental.h. This means that Audio Cache will not be in releases but anyone building Audacity will be able to uncomment the define and so make the Audio Cache checkbox appear in Directories Preferences. This will allow us to work on correcting the issues in this feature.

That is what I wanted to hear

steve wrote:

PGA wrote:You are assuming the digits are decimal digits. But if the letter "c" is a valid digit, these must be hexadecimal digits.

The normal sequence for the project data starts at d00 (which is the salient point being discussed).
The numbering is indeed hexadecimal, so will (probably, I've not tested a big enough recording) go up to "dff".
As Gale wrote, "It should not get to d0c before having got past d99."

The sequence should go,
d00, d01, d02, d03,...d09, d0a, d0b, d0c, d0d, d0e, d0f, d10, d11...

I knew these were hex but forgot what that meant in terms of ordering - I did tell you I was not a mathematician.

So yes, it would get to d0c (the 13th folder) before d99 (the 154th folder).

And yes, as there can only be 256 "d" folders, the last one in any "e" folder must be "dff".

Having just made a fairly lengthy recording to RAM I see the order I would expect from http://ascii.cl/conversion.htm :

d00 d01 d02 d03 d04 d05 d06 d07 d08 d09 d0a d0b d0c d0d d0e d0f d10 d11...

But from:
I would still like to know if you had pressed Stop when you produced that image (which would have copied the ,au files in the folders from RAM to disk) or Pause (which would not have copied to disk).

There are three major problems in that image.

* Where are the d00 through d05 folders? DId you delete them? Even if you had only pressed Pause, Audacity should still have created empty e00/d00 through d05 folders on disk, ready to populate them with .au files on Stop.
* Where is the d0c folder that should follow d0b? It seems it wasn't written.
* The big one: d06 (which should have been written before d07) being written at the same time as d0e, and what is more, at 15:34 instead of 15:37 which would have been expected from the previous folders. That is, unless Audacity does this when it starts writing to disk when system RAM falls below the Audacity minimum. I'll test, but I don't think so.

Do I assume you crashed at or after 15:34?

And where Dennis said elsewhere

but NOTHING in directory .d01

that would only be correct if you had pressed Pause rather than Stop at that time AND d00 had actually been writing to disk instead of RAM, which would be very odd in itself.


Gale

otwo_pipes wrote:Start Audacity as normal
Do not recover project when Audacity prompts for the required action to continue
Start recording: however, recording does not start. Normally, when I start recording, the play and record meters are active, the audio waveform capture screen is running and the time stamp and vertical cursor are scrolling. The only sign of recording trying to start is the vertical cursor blinking but not moving. The only way out of this scenario is to close Audacity and re-boot the PC. I have also seen this fault, on numerous occasions, when recording with the audio cache disabled.
(I suspect deleting the Audacity temp directory, after closing Audacity may also work but one thing at a time please.)

I would be surprised if the temp directory was relevant. I occasionally saw the stalled recording and blinking cursor on my older Win 7 machine and the temp directory was empty.

otwo_pipes wrote:Start taking notes regarding results from the previous and the following tests.
Close Audacity
Re-boot
Start Audacity as normal
Start recording
This recording crashed at 19:09
Repeat above procedure and Audacity crashed at 19:24

I was monitoring the directory writes on the second crash and I suspect Audacity crashed when trying to write a .au data file to disc. I was watching the .au files being created and it seemed to be the correct timing for the next .au file to be created. This is only a gut feel and no times were measured however this may be a useful pointer. I was monitoring the available memory at the time of the third crash and this was oscillating around the 85-105 MB mark even though the audio cache was set to 16MB.

The data results I have gleaned from the three crashes is as follows:-
Directories were correctly created in sequence of d00 - d06
The 1st crash had 18 .au files in the d00 directory
The 2nd crash had 20 .au files in the d00 directory: all files were 780kB apart from 2 files which were 318kB. (Why are there two files of size 318kB when Audacity crashes? I could understand all files being 780kB and one file was a different size but two files of 318kB, seems odd to me.)
The 3rd crash had no .au data files in any directory.

Let me know your views ideas and document any tests you require me to perform.

I think we know that writing .au files to disk when it should not be doing so doesn't always happen when there is a crash.

At the speed Audacity would be writing .au files at 24 x 192, this suggests that the first two crashes occurred within about 10 or 15 seconds of starting to write to disk.

Can you take a look at my post above http://forum.audacityteam.org/viewtopic ... 62#p185462 and offer any comments?

The only new discovery I have made so far on my newer Win 7 machine is that Audacity is ignoring high Minimum Free Memory settings of 2500 to 2900 MB and carrying on writing to RAM regardless when available RAM falls below that level.



Gale

The only new discoveries that I have made are:

• An apparently "out of sequence" data folder structure can be created by discarding Undo History.
• I can set stupid numbers for "Minimum Free Memory", including zero, negative numbers, really small numbers and really huge numbers (12345678912345678123456789, which then appears as 1.23457e+26)
• Changing the location of the temp directory makes no difference to the behaviour on my machine (Audacity ignores the option and always writes to disk).