Alternative recovery tool

[pjaytycy]

Hello,

Yesterday I recorded an interview, but I shut down my laptop before audacity was finished saving.
I noticed a lot of .au files were present, but the .aup file was empty. So, I decided to try to recover this.
I found the recovery tools on the wiki, but these seemed unnecessary complex (read all .au files, convert and append to one .wav file, ...).
I realized the only thing I need is to recreate the .aup index file, which is in XML format.

Find my python script for recovery attached here.
Notice, it was written purely to recover my interview, based on the .aup files for similar interviews I had done with the same settings. So, a lot of the data in the XML file is hardcoded to those values.

Some description / documentation:

create_aup() is the main function, it requires a directory name and a project name.
=> at the end, it will generate the file "<directory>project.aup"
=> list all files which match the pattern "<directory><project>_datae*d**.au"
=> sort by last-write / modification time
=> split in odd/even list (left/right channel)
=> add the audacity XML beginning structure
=> add wavetrack for left channel
=> add wavetrack for right channel
=> add the audacity XML ending structure

for each wavetrack, I do the following:
=> try to add the .au from the correct channel
=> if that fails, try to add the .au from the other channel
=> if neither channel has a good .au file, just skip this part (there will be 6 seconds missing)

This approach is good enough for me, but ofcourse the result is not perfect:
--> When listening with a headphone I can clearly hear sometimes when one of the speakers jumps from one channel to the other channel.
--> Some parts were missing in both channels, so here and there 6 seconds are skipped (they are NOT replaced by silence).

Why I like it better than the alternatives (creating a .wav file):
--> You can just open the .aup file directly in Audacity, without having to fiddle with 2GB big .wav files
--> It uses almost no memory, is very fast, doesn't write much to the harddisk, should be able to handle any size of data.
--> It uses only 3 standard python libraries : glob, os, struct. No other dependencies.

Probably this can help some other people as well.

[kozikowski]

Terrific idea. I've said that the only reason a tool like this doesn't already exist is a developer has never done what you did to his daughters grade school play.

Someone will immediately point out this will fail if you try it after editing. If an edited projects goes into the mud, that's pretty much the end of the world.

Koz

[Gale Andrews]

Thanks for posting.

I take it you are on Linux? Ext 3 file system? The channels being in wrong order in a stereo recording is always the problem.

Is it python version-dependent? I get no aup file written in Python 2.7.5 on Windows (but no errors either).


Gale

[pjaytycy]

I developed it for Python 2.7.2 on Windows 7, with an NTFS filesystem.
Can you post how you run the tool and any output it generates?

[Gale Andrews]

I developed it for Python 2.7.2 on Windows 7, with an NTFS filesystem.

OK so as NTFS has timestamp granularity of 0.0001 seconds you should be able to retrieve the AU file timestamps accurately enough to string them together in the correct channel order.

I don't know how to do it in python but in PowerShell, in the directory where the AU files are, the command is:

Code: Select all

gci | sort -property creationtime,name | foreach-object {$_.name + " {0:yyyy-MM-dd_HH:mm:ss.fff}" -f $_.creationtime}

Can you post how you run the tool and any output it generates?

I put aup_recover.py in a folder called "aa_data" containing four AU files. I opened a command prompt at the aa_data folder then ran:

Code: Select all

python aup_recover.py

No output. I don't know python as a language.


Gale

[pjaytycy]

Ok, that is behaviour as expected. The initial version was not really a stand-alone script. You had to do something like this in a Python shell:

Code: Select all

>>> import aup_recover
>>> directory = r"D:Datainterviews"
>>> project = "MyFirstInterview"
>>> aup_recover.create_aup(directory, project)

However, this is indeed not userfriendly. I have added some code so it is now an interactive script. You can run it like you did and it will aks you for the directory and then present you a list of projects it found in that directory. It will also refuse to overwrite existing .aup files (so you need to rename/backup these manually first).

This second version is attached to this post. However, as this starts getting "versions", I've put it on github as well: https://github.com/pjaytycy/aup_revocery

[Gale Andrews]

Thanks, this second version tested well for me.


Gale