Thanks for your efforts.
No problems, it is my pleasure. More to the point, it is good to keep my hand in now I am retired
However I can tell you that each dXX directory inside the eXX directory is written to until the d directory contains 256 .au files, then the next "d" directory in the sequence is populated with .au files.
This is NOT what I have seen. I have not counted these files as I did not think it was an issue though I think the number of .au files in each directory varies and last night I think I was seeing as low as 5-10 files (I think) but again, it was late and I was not looking too closely however you may find my latest screen shot interesting as this shows there are not always 256 files per directory. The screen shot is from an LP rip without a crash. The main picture shows the timing of each directory 'date modified' and the small pictures show the contents of each directory and date created. Look very carefully at these time and dates. Is there a clue here?
I have had a bad day ripping LP's and Audacity has not been well behaved, no crash but a right pain. After many 'fixes' to Audacity I now have your 256 files per directory (and one run that should have crashed with the Audacity settings but it did not). Don't you just love systems. At least I can prove what I have said re there not being 256 files per directory otherwise you may have started to dismiss my input.
I did not make an issue of it last night but the 24-192 capture had files written in directories .d00, .d02 etc. but NOTHING in directory .d01 and these Audacity settings did not cause a crash. I do not know if this is relevant but everything is worth mentioning.
Each .au file is 1 MB in size by default.
This is not what I seeing, my files are 780MB and not, as you say, 1MB. |I don't know the relevance of this but I thought it was worth mentioning.
Thus assuming Audacity was actually writing data, then the greater the sample rate and bit depth, the sooner each directory will be filled and the next one populated.
The greater the sample rate the quicker the directories are written; this is agreed and explains the variation in time to crash being sample rate dependent. At the moment I do NOT agree that bit depth alters the rate of directory writes but please remember, my results of last night are based on a sample test of one. This is not statistically significant so I am looking at this now.
otwo_pipes wrote:I think it is very interesting Audacity is creating these directories but is not writing to them but is writing somewhere. The amount of available system RAM, during an audio capture, decreases to a certain point and then continually oscillates i.e. available memory rises, falls rises etc. until Audacity crashes.
1. Does the certain point before which Audacity uses RAM ever vary with its "Minimum Free Memory" setting? You seem to be saying not? If not, I think it may be an idea to try a 24x192 capture with minimum settings between 16 MB and 600 MB.
The 'certain point' varies dependant on the setting of the audio cache depth.
If the audio cache is set to 16MB then system RAM is used until only around 95-100MB of the system RAM is free then the oscillations start followed by a crash some time later. If the audio cache is set to 100MB then RAM is used until around 300MB of RAM is free then the oscillations start followed by a crash some time later.
The some time later is dependant on the time between each make directory. This time is dependant on the sample rate and may or may not be dependant on the number of bits in the sample. I know the time could be dependant on the number of bits in the sample but so far I have not seen this. The crash occurs after the make directory d06 and before make directory d07, about one third to half of the make directory time interval after directory d06 is made. As I said, this time is definitely dependant on the sample rate and may be dependant on the bit rate.
2. On what basis do you believe that when you reach this certain point, Audacity starts writing to disk, given it is not writing in the temp folder? The names of the .au files are random, but perhaps it would be an idea to search for .au files by date created, to find out where they are.
The reason, for my assumption, is very simple.... When capturing data at 'Rate' bytes per second the storage required is the 'Rate' multiplied by Time (in seconds) bytes. When this number exceeds the amount of RAM used, since the start of recording, Audacity must be writing somewhere and the VRAM, allocated to Audacity by Windows is continuously increasing from the start of capture. As I said, I now assume Audacity is writing to the Virtual Windows RAM. Windows could of course be allocating VRAM to Audacity and Audacity discarding the captured data. I have not looked for the .au files, I just assumed. Sorry for this assumption and I like your question.
3. Are you sure that after the certain point, Audacity is not writing to RAM as well as to disk, given Windows is continuing to allocate more and more virtual memory to Audacity? Might the available memory not be depleting further due to swapping to the page file?
I suspect, and always have, that Audacity is writing to both RAM and VRAM from the START of audio capture when audio cache is enabled. I know this is not how it is intended to operate but this seems the only explanation for what I am seeing. Remember, I do not have a development tool-set, only the Windows Task Manager tools. Let's assume Audacity settings are such they will cause a crash. There are numerous choices of settings but the results are always the same aka:-
The system RAM is continually decreasing to the oscillatory value, as explained before, AND the VRAM allocated to Audacity by Windows, or whatever, is continually increasing. These actions occur from the start of recording until the crash. Maybe the VRAM point is very significant. Let me ask the question: If Audacity/Windows has over 1GB (and maybe as much as 1.5GB) of system RAM available why should Windows/Audacity increment the VRAM allocated to Audacity from the start of the audio capture? This behaviour does not make any sense to me. Let me run it past you again:-
The system RAM available to Windows is continually decreasing from the start of recording until the critical available memory, which is dependant on the audio RAM cache depth:-
If the cache is 16MB the system RAM decreases to about 100MB
If the cache is 100MB the system RAM decreases to about 300MB
then the available system RAM oscillates around this figure whilst the VRAM allocated to Audacity has continued to increase from the start of the audio capture
I can only assume Audacity is writing to RAM and VRAM but cannot be certain because projects are never recoverable after a crash.
4. If you set the minimum to 600 MB are you getting RAM used until system RAM falls below 600 MB, then disk writes - i.e. does the procedure work as intended if the minimum is high enough?
I do not think so but this one is difficult to answer with the tools I have available. I think the RAM is used until somewhere in excess of 1GB is in use, but never as much as 1.4GB (I have 2GB RAM) and then the oscillation starts. The oscillations are from around 1GB to 1.3GB; it was late and I am hazy on this point. What I am certain of is the correlation between available RAM and the cache setting but it is only a correlation; the two are NEVER equal i.e. 16MB cache correlates to 100MB system RAM and 100MB cache correlates to 300MB system RAM. I think 600MB cache correlates to 1GB system RAM but I am not sure on that one. It was gone midnight when I finished capturing audio
otwo_pipes wrote:To progress I think we need to have independent confirmation of the above findings regarding the crash timing.
It's quite difficult to fix anything that the developers cannot replicate on their own machines. If the developers cannot reproduce it, we are looking at giving you a debug build of Audacity, then you installing Visual Studio and examining call stacks (even if you wanted to do that)
I have no problem trying to help, other than time. My priority is ripping my Vinyl collection to HD. I have around 800-1000 albums and it takes about one hour per album. This equates to........... a very long time. I am going to live in Thailand from October for 5-10 years. I will travel SE Asia on my Valkyrie motorbike using Thailand as a base... now I am digressing
Also, it is quite difficult trying to de-bug at a distance. I need to be accurate and not cause further confusion and to this end this update has taken many hours to write. I was also ripping LP and trying to sort other Audacity problems but that is another story. I have to read the FAQ first