Hello, thank you for your response. Which public server do you suggest I use.
I have used dropbox and google.
You’ll need to provide access.
Caprice,
OK, I have been looking at your file. As I said before, “these particular error messages look very nasty - it is quite possible there is nothing that can be done.”
I have looked into your particular .aup3 file with all of the Audacity tools I have at my disposal - none of them will get anywhere with your file. Audacity makes use of an underlying sqlite database where it stores all of your data. In this case, the underlying sqlite process returns an error 11 = Corrupt file, and refuses to do anything else.
I have also explored several sqlite utilities that claim to have some success recovering databases from these errors. None of these utilites have proved fruitful.
However, when I look at the raw data buried within your .aup3 file, I can see what appears to be chunks of audio data interspersed between unintelligible and damaged control data. When I take your .aup3 file and import it as RAW 32-bit float data, Little-endian, I can almost see (and hear) your audio.
I am thinking if we can remove the control data from this result, we may be able to recover some/much of your data.
For this reason, I am asking a Nyquist developer if he can assist with a program that might be able to remove these data points. It may be of general value. Stay tuned.
It is easy to remove all envelope control points IF you can open the project in Audacity (“Extra menu > Scriptables I > Set Envelope”).
Apply to the entire project and all envelope controls points should be removed:
If the project won’t open, then how do you access the audio data?
Thanks. No. Not envelope control points. I mean actual data points or samples. Sorry I didn’t make myself clear. So the file would actually end up a little bit shorter. (It is too long to begin with).
So looking at the screenshot, the invalid data point is not only wrong, it is extra and needs to be removed altogether. The invalid point should be treated as no data, and everything to the right needs to be shifted one sample point to the left. I don’t know if you can do this in nyquist.
It can be tricky.
If the “invalid data point” is a valid 32-bit float value, then:
- It’s unlikely to be a problem
- You could force it down into a valid range by clipping the audio to +/- 1.0
- A script could probably be written to export as a valid WAV file (with the invalid data points removed)
- For short tracks (less than about 30 minutes, mono, @ 44100 Hz) it’s probably not too difficult to remove the “invalid data points” with a conventional Nyquist “Effect” plug-in.
Option 3 is not simple for long tracks, but I think it would be possible.
The bigger problem comes with “invalid data points” that are not valid 32-bit float numbers, aka "NaN"s (https://en.wikipedia.org/wiki/NaN)
I did some work with Nyquist a few years ago to help Nyquist deal with NaNs safely - I’ll need to test to see what the current situation is.
Thank you, Steve. Thanks for pointing out the issues.
Mono tracks only, and quite slow.
Tested with 1 hour audio selection (took about 2.5 minutes) with 64-bit Audacity on Linux:
This code replaces any invalid / out of range samples with silence:
;version 4
;type process
(setf step 100000)
(defun process (sig)
(let ((out (s-rest 0))
(ar-out (make-array step))
(sr *sound-srate*)
(tn (/ step *sound-srate*)))
(do ((ar (snd-fetch-array sig step step) (snd-fetch-array sig step step))
(i 0 (1+ i)))
((not ar) out)
(dotimes (j step)
(if (and (<= (abs (aref ar j)) 1) ;valid range
(= (aref ar j) (aref ar j))) ;not a NaN
(setf (aref ar-out j) (aref ar j))
(setf (aref ar-out j) 0)))
(setf out
(sim out
(at-abs (* i tn)
(cue (snd-from-array 0 sr ar-out))))))))
(process *track*)
Thank you very much Steve, your program was very helpful and illuminating. I see you found a solution to the NAN issue. I had been hoping we could find a Nyquist solution as this could be something anyone could use easily.
Other than what you mentioned, your program has two limitations: (1) It zeroes out the erroneous data, rather than remove it, and (2) it terminates without an error message when a very large section of data is selected.
Nonetheless, there is a distinct flaw in my concept. While the program does remove most of the noise that occurs every 1/3 of a second or so
, there is still noise at 14 second intervals that needs to be addressed
. Back to the drawing board for me and thanks again for you assistance.
Before running your program:
.
After running your program:
.
You can see that a lot of the data has been cleaned up…
Both of those could have been fixed if it was sufficiently useful to spend time on it
Can you tell if the bad samples are “extra” samples, or, if they were all removed, would there be bits missing from the sound?
Yes, I know, - but I think we have bigger fish to fry.
Yes, they are “extra” samples. If they were all removed, then the sound would be perfect - except for the possibly bigger issues I have outlined above.
It looks like the remaining “bad” data is mostly much higher amplitude than the “good” data.
If you want to see what’s left after removing all samples that have an absolute value > 0.5, change this line in the Nyquist script
(if (and (<= (abs (aref ar j)) 1) ;valid range
to
(if (and (<= (abs (aref ar j)) 0.5)
.
Thanks, but I think it is going to need a different approach. Importantly, we’ve got your mind churning.
Caprice,
Just so you know, I am still looking into your project.
So I’m making a little progress. I PM’d you a link.
Are you referring to this post: https://forum.audacityteam.org/t/audacity-failed-to-set-safe-mode/62388/4
or is this a new issue? If this is a new issue, then please start a new topic in the forum. If it is the former, I’ll comment there in a few days.
Hello,
This is a different issue. I will create a new topic
Thank you
Caprice,
I have uploaded a new, extraction from your Corrupted file. I have sent the link via PM.
It is pretty much the same but with far far less noise.