100% orphaned blocks? I would not call this program reliable

[charlieb000]

Im moving the 30GB recording to another drive so I have the free space to re-enact the original recording (blank recordings will NTFS compress better, it might allow >2x). This method only works if you choose the right bit rate, and assumes v2.0.2 names the files beyond 13.5h the same as v2.1.2 . I really would like to know how the names get chosen so I do not need to do this (one alternative is to download someone else’s 44100Hz AUP file - a download intended for everyone will need breaks every 256 entries, or audacity can generate a new one for the user). I have tested it with a 40minute recording and it works, so perhaps you can include this method in your manual as a recovery possibility, the only uncertainty is which way the the last piece of each channel goes since the last piece is named different.

None of these ("2.02", "2.12") are made by us.

Yes, 2.12 = 2.1.2. I still have not gotten used to extended version designations, I have seen too many old programs! (even this forum refers to audacity as being 2.x rather than 2.x.x) "This forum is for Audacity 2.x on Windows."

The "min" "max" and "rms" are amplitude levels.

Does it impact the product in any way??? Min and max do not seem to be used in 2.0.2 so if it is it will be less for me.
As for those recovery options, stereo is not perfect, I have sorted by date in XP-explorer and the order is messed up (compared to my method). I think that it is a little too hopeful to assume that the dates were correct anyway.

If you were intending to make a 10,000 km car journey .... reliable power (preferably UPS)... simple like zooming ....25 hours of stereo audio at 44.1 kHz contains around 8,000,000,000 samples)...

*UPS = laptop
*I was recording internet radio. I actually am planning to go on a trip soon, needed something to listen to (new years is a good time to record since they make sure the music is better). I was not intending to make a 25h recording, but I stayed too long at a new years day party.
*Zooming should not need THAT much processing power, my screen is only 1920pix wide (minus whatever it is for the audacity UI) - doesn’t it only need to average a few data spans for each pixel? Spans can be every 100th recorded magnitude (sized to appropriately catch all freqs)… Seeing 25h, you really cannot see much detail anyway so it really can be skimpy - Perhaps to the point of having only a representative magnitude for each piece, remember, I had >30,000 pieces, 30k calcs are easy for a computer (is that what the "rms" or "max" is for in the AUP file?). Alternatively, could use F5 to update the quality if the existing image is no good (Stretch existing drawing if the user pressed once or twice. If more, start redrawing after the user stops pressing zoom and stops scrolling). Perhaps have separate Q+ Q- and updateQ buttons to increase, decrease and zoom-optimise the quality, so its updated separate to the zoom, or allow quality-creep (slowly updates the quality for the zoom as the user works, with a preference for the places between existing details that show a dramatic change, so there is no sustained hanging - only a lag).

It would help if there was a plugin that searches for long (0.5s) silences at around a certain distance apart (ie 1-2h) and cuts it up (or I can leave it as one big rec), but I guess this is half the “fun” of using audacity.

[charlieb000]

it stops when the disk is full...

Audacity doesn't stop when the disk is full but carries on, corrupting the existing recording as it goes. It is a high priority for us to fix that.
Gale

I did come across that while trying to find a way to avoid moving the first recording off the drive, it doesn’t save if there is a block missing! (i deleted some as it was recording). It only creates some folders. The solution for that seems kind of simple… let it save anyway and let the user deal with the missing blocks when the project is loaded. It is possible the user only needs to measure the time between two sounds/events that are way too far apart, and he can still save the timing even if the drive is too small, and be able to read the parts he moved off the drive (still aligned to the appropriate time) after changing disks and reloading.

[Gale Andrews]

Im moving the 30GB recording to another drive so I have the free space to re-enact the original recording (blank recordings will NTFS compress better, it might allow >2x). This method only works if you choose the right bit rate

Do you mean sample rate, or strictly bit rate (which for PCM is sample rate * bit depth * number of channels)?

assumes v2.0.2 names the files beyond 13.5h the same as v2.1.2 . I really would like to know how the names get chosen

The file names are chosen randomly. If you don't edit the recording, and you have access to sufficient timestamp granularity (which you do with NTFS file system in the recommended recovery method), the randomness does not matter because the file timestamp order matches with the timeline order in Audacity.

I have tested it with a 40minute recording and it works, so perhaps you can include this method in your manual as a recovery possibility

I have no intention of doing that without thoroughly testing it, though I have heard of people attempting it. We have two known working methods including the python script I mentioned, so it is not a priority to add a method which is likely to confuse most users or be error prone, given they are directly manipulating the AUP file.

How for example are you changing the file names of the AU files in the newly recorded AUP file (the one to be used for recovery) to the names of the AU files in your original data?

None of these ("2.02", "2.12") are made by us.

Yes, 2.12 = 2.1.2. I still have not gotten used to extended version designations, I have seen too many old programs! (even this forum refers to audacity as being 2.x rather than 2.x.x) "This forum is for Audacity 2.x on Windows."

I agree with you about the "2.x" should be "2.x.x". I'll change it when I get time.

The "min" "max" and "rms" are amplitude levels.

Does it impact the product in any way???

That is for you to test if you want to use this recovery method instead of the officially recommended one. I think there could be subtle problems at least. I think the "rms" in the AUP file is only used in the "Contrast" effect.

As for those recovery options, stereo is not perfect, I have sorted by date in XP-explorer and the order is messed up (compared to my method). I think that it is a little too hopeful to assume that the dates were correct anyway.

You mean the xplorer2 Pro routine in http://manual.audacityteam.org/man/reco ... very_tools? I have not tested it many times but it worked perfectly for me. It does rely on you having an NTFS file system, not FAT32.

I don't think you have said what version of Windows you are using.

Zooming should not need THAT much processing power

The amount of data to be gone through makes a difference.


Gale

[steve]

*Zooming should not need THAT much processing power, my screen is only 1920pix wide (minus whatever it is for the audacity UI) - doesn’t it only need to average a few data spans for each pixel?

Audacity needs to know the peak and RMS level for each pixel. A cache system is used to speed this up, which includes, among other things, the Min, Max and RMS values that are written to the XML data. In order to populate and update the caches, the audio samples have to be analyzed, so at some point Audacity has to go through all 8 billion samples, recording the minimum and maximum per block and calculating the RMS per block. Once the caches are populated, drawing the waveform should be fairly quick.

[charlieb000]

Do you mean sample rate, or strictly bit rate (which for PCM is sample rate * bit depth * number of channels)?

I think I changed the “project rate”at the bottom before I began recording (so the answer is “whatever that box changes”). I have only changed 44100 to a lower figure in an attempt to reduce the disk space required for the re-enactment, only to find the file names were different, so I had to keep it at 44100.

The file names are chosen randomly. … How for example are you changing the file names of the AU files in the newly recorded AUP file (the one to be used for recovery) to the names of the AU files in your original data?

The “random” file names are the same every time a new project is recorded, so I do not need to rename anything (providing the “project rate” is the same). Since the last message I posted, I have noticed I can overwrite the d** folders in the temporary folder with the data I desire to recover even while I am recording, and when I zoom out I can see the recovered data appearing at the start (this was another idea I had to save disk space - the 16x d0* folders cover 3.5hrs, so I can do a replace/move that often to keep it simple). If it suddenly did become random, It would be easy to notice because it should become gappy (I hope I am not breaking it by doing the replacement). if I continue, I will have to not lose any piece I remove - else it would forbid saving (and I would have to force-close audacity to do it's recovery, etc). I still will have to make sure it doesn’t modify any of the pieces that have already been recorded, that’s all.
First 10 for ch0: e0000005 e0000317 e000095c e000059a e0000d2a e00002c9 e0000b5e e00004dd e0000176 e000025b
First 10 for ch1: e0000904 e0000cf0 e00007ad e0000e55 e0000bf2 e0000dbe e0000837 e000003d e00005d4 e00002a7 @ 44100hz

That is for you to test if you want to use this recovery method instead of the officially recommended one. (This is Gale's response for “what is rms/min/max in the AUP file”)

I thought you would know the purpose of these. It would be good to have tools to subtract/add/etc the waveform so I can exactly see (Cannot really check if there are any at this moment because I am recording “nothing” right now and I am not allowed to have another window). I once tried to have two cancelling waves but I could still hear it when I played it back, so listening to it inverted to hear a difference is not going to work well anyway. So what I will do later is produce some AUP files that have really messed up values and see how/if it breaks anything.

xplorer2 Pro

Erm, by “Xp-explorer” i meant “windows XP’s explorer”, if ms is recorded in the date stamp, windows explorer is certainly not reading it to perform the sort.

Audacity needs to know the peak and RMS level for each pixel.

The peak I understand, it’s the scale. RMS, I do not – take your mains voltage for example, that number is the RMS value – the average. 120V, 240V, 415V (three phases of 240V, 120° apart), 480V (split phase, 240V each). 240VDC is the same power as 240V AC power except AC can rise to 339.4V, the only possible use I can see is perhaps for non-parity error checking. Speculation: I do not see how knowing the average will assist in obtaining a value of a specific level - except if you are using RMS to scale the data as well, it will work but I do not think it is unnecessary, it would change the quality of the scaling depending on what is in the recording (especially DC bias), rather than only its peak volume, and a peak can go above the average by many times (But since it is working as is I shouldn’t be complaining. If it is that way it would hinder accuracy further if I happen to lose the “key”). It is possible the RMS is only used for the amplitude display, that is what the second amplitude is for in the editing area is (one is darker blue, other is lighter blue), so maybe something to do with that.

As for the status of the re-enactment, well, after 5 hours, I accidentally clicked [X] on the corner of the window (because I was clearing the screen) and that was sufficient to cause it to stop recording without my permission (similar to it automatically selecting “no” to “do you want to save your changes”). Then after an additional 3h, i was reminded that I forgot to make sure the battery was in the computer when the power went off (yes, I think it is funny too, since it was mentioned earlier!). I keep it out because I know every time i charge it it shortens its lifespan (and in storage it degrades more if it is holding more charge or there is more heat in the storage location) – since I cannot switch off the charger in the laptop (at maybe 30% full), I take the battery out. And I have restarted the recording again this next morning, where I discovered what happens when I replace the temp recording files.

[Gale Andrews]

The file names are chosen randomly. … How for example are you changing the file names of the AU files in the newly recorded AUP file (the one to be used for recovery) to the names of the AU files in your original data?

The “random” file names are the same every time a new project is recorded

Sorry, but that is simply untrue.

I have noticed I can overwrite the d** folders in the temporary folder with the data I desire to recover even while I am recording

Alright so presumably you don't care about saving this project, which on reopening would have missing and orphan block files.

xplorer2 Pro

Erm, by “Xp-explorer” i meant “windows XP’s explorer”, if ms is recorded in the date stamp, windows explorer is certainly not reading it to perform the sort.

We already make it clear in http://manual.audacityteam.org/man/reco ... very_tools that Windows Explorer cannot sort accurately enough to recover a stereo recording with correct left/right channel allocation. xplorer2 Pro can do so, on an NTFS file system.

Again, the link above contains the official instructions (or you can use the python script I mentioned).

That is for you to test if you want to use this recovery method instead of the officially recommended one. (This is Gale's response for “what is rms/min/max in the AUP file”)
I thought you would know the purpose of these.

Steve knows more about this than I do, but I think RMS, min and max are also stored in the AU files themselves. Whether the AU or XML data gets used depends on the zoom level. He can clarify if he wants to.


Gale

[steve]

The “random” file names are the same every time a new project is recorded

Not unless you are using an ancient version of Audacity.

The current blockfile/directory scheme uses two levels of subdirectories - up to 256 'eXX' and up to 256 'dYY' directories within each of the 'eXX' dirs, where XX and YY are hex chars. In each of the dXX directories there are up to 256 audio files (e.g. .au or .auf). They have a filename scheme of 'eXXYYZZZZ', where XX and YY refers to the subdirectories. The 'ZZZZ' component is generated randomly. The XX and YY components are sequential. DirManager fills up the current dYY subdir until 256 are created, and moves on to the next one.

[charlieb000]

Sorry, but that is simply untrue.

Not unless you are using an ancient version of Audacity. ... The 'ZZZZ' component is generated randomly.

I am writing to tell everyone what I did to recover my data. If it is supposed to be random, you can help find what was needed to break it so other’s data can be recovered quickly (audacity would be able to do it by itself, fast), or find some circumstance that is needed for this so that a few people can recover their data more easily. I do not see any GOOD reason why you need to randomise it (unless hindering data recovery is a ‘good’ thing? or preventing overwriting of the data?), audacity only needs the filenames to be unique - they can be jumbled up or deleted later as the user edits it. Since this is obviously good enough for me, it can be good enough for everyone. If some people want the names scrambled, let it be an option in the options menu or /command line.

Alright so presumably you don't care about saving this project, which on reopening would have missing and orphan block files.

I did it this way because I backed it up onto my USB drive and didn’t want to wait for it to copy it back. All files moved displaced the existing files. I saw no errors - since the 2 expected orphans that were present were ignored after I pressed save, the temporary folder containing them didn’t get deleted.

As for RMS, MIN and MAX it has no effect on the output (I tested it on the first file on one channel). I expected you guys to be able to tell me their purpose easily since you recently changed the way the AUP file is saved (between 2.0.2 and 2.1.2) so the MIN and MAX is recorded differently. Obviously none of you had a hand in that. If it has no purpose you can still use them to help with zooming as I wrote earlier (or remove them).

This is the last message I needed to write. Don’t expect further replies.

[Gale Andrews]

As for RMS, MIN and MAX it has no effect on the output (I tested it on the first file on one channel). I expected you guys to be able to tell me their purpose easily since you recently changed the way the AUP file is saved (between 2.0.2 and 2.1.2) so the MIN and MAX is recorded differently.

We did not change that, or we don't understand what you mean.

2.0.2 first block of generated tone

Code: Select all

<waveblock start="0">
<simpleblockfile filename="e0000bfe.au" len="262144" min="-0.8" max="0.8" rms="0.565686"/>
</waveblock>

2.1.3-alpha first block of generated tone

Code: Select all

<waveblock start="0">
<simpleblockfile filename="e00002cd.au" len="262144" min="-0.8" max="0.8" rms="0.565686"/>
</waveblock>

This is the last message I needed to write. Don’t expect further replies.

I agree. We're just talking at cross-purposes.

The moral of the story is clear. Download Audacity only from http://www.audacityteam.org/download/ and keep it updated so you have the latest bug fixes.


Gale