Memory leaks in Nyquist or the Lisp environment?

[Paul L]

Some of my experiments in Nyquist programming are being frustrated by seeming memory leaks.

I don't have a small code example to demonstrate it, but let me describe what I'm doing.

Iterate over fft frames. (Typically a 2048 window and a 32 skip.) Calculate a certain value based on successive frame pairs. Use snd-from-array and other Nyquist functions to avoid inner loops in Lisp. The calculated values become the samples of another sound using snd-fromobject. A variable is needed to remember the "sound" from the current frame as coming from the previous frame on the next iteration.

If I try this on a big sound over 8 minutes long, I watch memory usage climb until it exhausts my computer and Audacity just crashes.

Inserting calls to (gc) does not help. If I cancel the calculation of the effect midway, memory use does not drop down again nor does it if I call (gc) from the Nyquist prompt.

I have written similar code that makes calculations from only one frame at a time with no need to remember the previous frame or the "sound" derived from it and I do not find similar problems though there is similar use of snd-prod and snd-avg in the calculations.

[steve]

Memory management for Nyquist in Audacity is far from ideal. One of the major problems is that Nyquist does not read track audio data directly from disk. A well known example is normalizing an audio track:

Code: Select all

(setq newpeak 0.8)
(mult s (/ newpeak (peak s ny:all)))

The problem here is that (peak s ny:all) reads all of the selected audio "s" and is not released from memory until the multiplication is completed, thus normalizing audio in one pass is limited by the amount of free memory. There's some discussion about this issue here: http://sourceforge.net/mailarchive/mess ... d=18401605 (note that this is dated 2008 and some of the information no longer applies.
There is also a longer exposition of the normalization issue and a two-pass workaround, written by Roger Dannenberg, here: http://www.cs.cmu.edu/~music/nyquist/debug-plugin.html

A one pass workaround for the Normalization issue is:

Code: Select all

(defun get-signal ()
  (let ((local-s s))
     (setf s nil)
     local-s))

(let ((pk (s-save (get-signal) "temp.wav")))
  (mult (/ pk) (s-read "temp.wav")))

but the downside of this is that it leaves a copy of the sound (temp.wav) on the hard drive, which is not really desirable.

If I try this on a big sound over 8 minutes long, I watch memory usage climb until it exhausts my computer and Audacity just crashes.
Inserting calls to (gc) does not help. If I cancel the calculation of the effect midway, memory use does not drop down again nor does it if I call (gc) from the Nyquist prompt.

If you run your code on, say, a 6 minute selection, I presume that you see memory usage climb quite high. Is the memory released when the script completes?

[Paul L]

Yes, I just tried with just one minute and watched Audacity's memory usage in Task Manager climb from about 46 M to 683 M and not drop back even after the computation completed.

Is the process taking more memory but not really leaking it? Does its memory footprint not change if I undo and repeat?

I tried that, and memory usage climbs at once from that plateau to a higher one.

Now some of my complicated calculations do this and others don't. I haven't isolated what the difference is. All these experiments involve lots of tiny and temporary "sound" objects typically 1024 or 2048 samples long.

I know you can write leaky code in Lisp that fails to remove references to variables and so fails to make them collectible. I don't believe I do that. Furthermore, if the entire Lisp environment is destroyed when the effect completes, I would think there would be delayed reclamation then, but I don't observe that.

[steve]

Yes, I just tried with just one minute and watched Audacity's memory usage in Task Manager climb from about 46 M to 683 M and not drop back even after the computation completed.

It 'should' drop back on completion, so if it isn't doing so then I think it would be worth trying to find out why.

If you can narrow down the problem, or an example of the problem to a reasonably short piece of code, please post it and we can take it from there.

[Paul L]

More experiments done with selective commenting-out of the complicated calculation I am attempting.

The surprising answer is that the culprit is innocent looking snd-add, but not just any use of it. It seems a necessary condition for the excessive memory use.

What I'm doing: Use snd-fromobject to calculate a "sound" of values derived from fft frames.

For each frame, use snd-from-array to make a "sound" with start time 1 and rate 1.
Add (snd-const 0 0 1 1) to that.
Then make a power spectrum by multiplying the sum sound by itself, and using snd-avg to average successive pairs of samples.
Then, other blah blah blah using snd-avg and diff and rms.

The surprising thing is that removing the bolded step, though it makes my calculation meaningless, makes the huge memory leak go away.

This even though the little one-sample sound I am adding is a constant lifted out of the repeated calculation, not reconstructed each time.

I don't yet have a simplified piece of code demonstrating it.


Now perhaps I can work around this by using snd-xform to eliminate the DC component of the fft frame, rather than trying to pair it with another sample inserted before it. I probably don't care about it in nicely behaved sounds without an offset.

[Paul L]

I tried what I described with snd-xform, and got only slightly different numerical results without the memory leak.

[Paul L]

I have emailed Roger Dannenberg about this problem and he has reproduced it. It appears that addition of sounds with domains that do not overlap results in memory leaks.

[steve]

Thanks for the update Paul.

Probably related, I found that the following code causes a crash:

Code: Select all

(do ((i 0 (1+ i)))
    ((= i  100000) s)
  (setf s (sum s (snd-const 0 0 *sound-srate* 1))))

[Paul L]

Do you see climbing memory usage before a crash? On Windows I watch Task Manager. I have caused memory exhaustion which makes Audacity crash. But could it be crashing otherwise?

[steve]

Do you see climbing memory usage before a crash?

I see "some" climbing memory, but only about 60 MB, then crash.