How to capture envelope from one track and apply to another

Using Nyquist scripts in Audacity.
Post and download new plug-ins.

If you require help using Audacity, please post on the forum board relevant to your operating system:
Windows
Mac OS X
GNU/Linux and Unix-like

How to capture envelope from one track and apply to another

Permanent link to this post Posted by dougherj » Thu Nov 19, 2015 5:54 pm

Hi,

Looking for some help / direction on creating a Nyquist plug or code in the prompt to do the following.

I have a stereo binaural beats track. I would like to mix it with various music files (2nd track), but I want to have the binaural track have an envelope that follows the music track. Part of the idea for binaural beats (according to some) is that it should be barely audible behind the music. For the raw beats track, adjusting the volume either completely hides it during loud music passages or makes it too noticeable during quiet passages.

I can achieve my goal (kinda) by manually applying the envelope tool on the beats track, eyeballing the music track. This is real tedious for a long (1 hr) music track and less than perfect.

Is there an obvious way to achieve this in Audacity? If not, any suggestions on coding this in Nyquist?

John
dougherj
 
Posts: 2
Joined: Wed Sep 22, 2010 10:50 pm
Operating System: Please select

Re: How to capture envelope from one track and apply to anot

Permanent link to this post Posted by Trebor » Thu Nov 19, 2015 10:59 pm

The magic phrase is "envelope-follower",
e.g. the second block of Nyquist-code here ... viewtopic.php?p=107790#p107790

Audacity has an effect called AutoDuck which does the exact opposite of what you want.

You could add, (generate), a white-noise track, AutoDuck that noise track using the music, then AutoDuck the binaural-beats using the AutoDucked white-noise track , ( then discard the noise track ).
Trebor
 
Posts: 3954
Joined: Sat Dec 27, 2008 5:22 pm
Operating System: Windows Vista

Re: How to capture envelope from one track and apply to anot

Permanent link to this post Posted by steve » Fri Nov 20, 2015 2:11 pm

Trebor wrote:The magic phrase is "envelope-follower",
e.g. the second block of Nyquist-code here ... viewtopic.php?p=107790#p107790

Unfortunately that code is a bit tricky to use because it requires the sound to be followed to be in the left channel of a stereo track, and the sound to apply the envelope to in the right channel of the stereo track. To use that code with stereo tracks (and binaural beats are stereo by definition) requires messing around splitting and rejoining the stereo tracks several times.

Here is an alternative Nyquist script that will work with mono or stereo tracks, provided the tracks are not too long (I've set the max selection length to 30 minutes as that tested OK). This code requires Audacity 2.1.1 or later.

Code: Select all
;type process
;control res "Time resolution" float "seconds" 0.1 0.01 10
;control mode "Follow peak or RMS level" choice "Peak,RMS" 0

(defun mono (sig)
  (setf sig (s-abs sig))
  (if (arrayp sig)
      (mult 0.5 (sum (aref sig 0)(aref sig 1)))
      sig))

(setf step (truncate (* res *sound-srate*)))
(setf op (- 2 mode))
   
(cond
  ((> (get-duration 1) 1800)
    (format nil "Error.~%~%Selection too long.~%~
                 Reduce the selection to 30 mins maximum"))
  ((< (length (get '*selection* 'tracks)) 2)
    (format nil "Error.~%~%This effect requires at least 2 tracks to be selected.~%~
                 The amplitude envelope is copied from the first~%~
                 selected track, then applied to subsequent tracks."))
  ((< (get-duration 0.5) res)
    (format nil "Error.~%~%The 'Time Resolution' should be considerably~%~
                 shorter than the length of the selection.~%~
                 The absolute maximum allowed 'Time Resolution' is~%~
                 half of the selection length."))
  ((= (get '*track* 'index) 1)
    (setf *scratch*
      (snd-copy  (snd-avg (mono *track*)  step step op)))
    ; return *track* to prevent audio from being released from *scratch*.
    *track*) ;*track*)
  (T
    (let ((env *scratch*)
          (offset (* res (/ (1+ mode) 2.0)))
          (initial-amp (snd-fetch (snd-copy *scratch*))))
      ; release *scratch* when we're finished
      (if (= (get '*track* 'index)(length (get '*selection* 'tracks)))
          (setf *scratch* '*unbound*))
      (mult *track*
        (sim
          (abs-env (pwlv  0 offset initial-amp))
          (at-abs offset (cue env)))))))


The track that you want to follow must be the first selected track.
The envelope detected in the first selected track is then applied to each subsequent track.
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)
steve
Site Admin
 
Posts: 45083
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: How to capture envelope from one track and apply to anot

Permanent link to this post Posted by dougherj » Fri Nov 20, 2015 8:41 pm

Steve

Thanks a lot! Perfect. I applied it, with the default 0.1, Peak to a 30 min sleep music file (which I mixed to mono first) and it created exactly the envelope that I wanted.

Now, I need to study your code and figure out what you did. Eventually, I'm going to learn Nyquist.

Thanks, again
John
dougherj
 
Posts: 2
Joined: Wed Sep 22, 2010 10:50 pm
Operating System: Please select

Re: How to capture envelope from one track and apply to anot

Permanent link to this post Posted by steve » Fri Nov 20, 2015 9:32 pm

Having done a bit of testing, I think that the code should be safe up to 200,000,000 samples (about 75 minutes at a sample rate of 44100 Hz), provided that the computer has enough available RAM.

dougherj wrote:Peak to a 30 min sleep music file (which I mixed to mono first)

You don't have to mix to mono for this effect. When calculating the envelope for a stereo track, the effect analyzes the average of the two channels.
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)
steve
Site Admin
 
Posts: 45083
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: How to capture envelope from one track and apply to anot

Permanent link to this post Posted by MisterHSP » Wed Mar 16, 2016 11:59 pm

Steve,

Thank you for providing this code. I have been using it in a similar manner as dougherj, but with affirmations rather than binaural beats (it makes them just barely audible). Thank you again, it is just what I needed.

I have a question, though. Is there a variable for how much it reduces the signal on the target track? I currently do a trial and error process with normalizing the target track then applying the Nyquist code. I can't make out in the code how it determines how much to dampen the signal on the target. I am trying to get the target track to be 15-18 db (on the VU meter) below the envelope track.

Am I making sense?

Thanks,
John (MisterHSP)
MisterHSP
 
Posts: 8
Joined: Wed Mar 16, 2016 9:07 pm
Operating System: Windows 7

Re: How to capture envelope from one track and apply to anot

Permanent link to this post Posted by steve » Thu Mar 17, 2016 1:14 pm

MisterHSP wrote:I have a question, though. Is there a variable for how much it reduces the signal on the target track?

For Nyquist plug-ins, the audio from the track that is currently being processed is called *track*.

The effect works in two stages:
1) The "source track" is analyzed in short blocks ("steps"), in which the peak level for each step is measured. Each of these measurements is added to a new "control signal" to produce a "sound" at a much lower sample rate than the original, that rides the peaks of the "source track".
2) The amplitude envelope (control signal) created in (1) is saved to a scratch-pad variable called *scratch*, which has a special feature of surviving in Nyquist from one track to the next.
3) When Nyquist processes the next track (the "target track") it applies the envelope (from *scratch*) to the audio from the target track (*track*) by multiplying (same as "amplifying") the track by the envelope.

The envelope follower is based on the snd-avg command: http://www.cs.cmu.edu/~rbd/doc/nyquist/ ... l#index684
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)
steve
Site Admin
 
Posts: 45083
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: How to capture envelope from one track and apply to anot

Permanent link to this post Posted by MisterHSP » Thu Mar 17, 2016 9:49 pm

Steve,

Thanks for the reply. I studied your answer, studied the code, and tried to cross-reference everything in the Nyquist functions and documentation you linked to. A couple of times, I thought I was almost there. But the thing I am searching for still eludes me.

Then, I got further confused when I took 59 seconds of a music track, duplicated it, and applied the Nyquist code to them. The duplicated (second) track was reduced in amplitude by about 12 db in the quieter portion and by about 5 db in the louder portion.

I still can't pinpoint what in the Nyquist functions is reducing amplitude in the second track... and what is is basing the reduction on.

Can you educate me on that?

Thanks,
John (MisterHSP)
MisterHSP
 
Posts: 8
Joined: Wed Mar 16, 2016 9:07 pm
Operating System: Windows 7

Re: How to capture envelope from one track and apply to anot

Permanent link to this post Posted by steve » Thu Mar 17, 2016 11:03 pm

This sets the code to run as a "process" type of plug-in (an "Effect"), and creates the two controls
Code: Select all
;type process
;control res "Time resolution" float "seconds" 0.1 0.01 10
;control mode "Follow peak or RMS level" choice "Peak,RMS" 0


This is a function that averages two channels of a stereo sound to create a mono sound
Code: Select all
(defun mono (sig)
  (setf sig (s-abs sig))
  (if (arrayp sig)
      (mult 0.5 (sum (aref sig 0)(aref sig 1)))
      sig))


This sets "step" to "Time resolution" in samples ("res" was in seconds)
Code: Select all
(setf step (truncate (* res *sound-srate*)))


The "mode" control has a value of 0 for the first choice (peak) or 1 for the second choice (RMS)
The SND_AVG function requires a value of 1 (op-average) or 2 (op-peak)
Code: Select all
(setf op (- 2 mode))


Test for some conditions (if something is true, do something)
Code: Select all
(cond


Just basic error checking
Code: Select all
  ((> (get-duration 1) 1800)
    (format nil "Error.~%~%Selection too long.~%~
                 Reduce the selection to 30 mins maximum"))
  ((< (length (get '*selection* 'tracks)) 2)
    (format nil "Error.~%~%This effect requires at least 2 tracks to be selected.~%~
                 The amplitude envelope is copied from the first~%~
                 selected track, then applied to subsequent tracks."))
  ((< (get-duration 0.5) res)
    (format nil "Error.~%~%The 'Time Resolution' should be considerably~%~
                 shorter than the length of the selection.~%~
                 The absolute maximum allowed 'Time Resolution' is~%~
                 half of the selection length."))



By this point, no errors have been found, and this is the first track, so grab the envelope and jot it down on a scratch-pad. Nyquist plug-in effects process one track at a time, and nearly everything is reset at the end of processing a track. The exception is "*scratch*" which can retain its value from one track to the next, so we temporarily put the envelope into *scratch*. (More about this envelope later).
Code: Select all
  ((= (get '*track* 'index) 1)
    (setf *scratch*
      (snd-copy  (snd-avg (mono *track*)  step step op)))
    ; return *track* to prevent audio from being released from *scratch*.
    *track*) ;*track*)


"T" means "true". This is the final part of the COND statement, so if none of the other tests have been "true", then do this.
Basically what we are doing is amplifying ("multiplying") the selected track (which is NOT the first track) by the envelope. Because our envelope does not quite start from the beginning of the track, the position of the envelope is padded a little at the start and shifted slightly to the right so that the peaks in the envelope will line up with peaks in the first track. Then, if this is the final selected track, we delete *scratch* so that it is not hanging around in RAM.
Code: Select all
  (T
    (let ((env *scratch*)
          (offset (* res (/ (1+ mode) 2.0)))
          (initial-amp (snd-fetch (snd-copy *scratch*))))
      ; release *scratch* when we're finished
      (if (= (get '*track* 'index)(length (get '*selection* 'tracks)))
          (setf *scratch* '*unbound*))
      (mult *track*
        (sim
          (abs-env (pwlv  0 offset initial-amp))
          (at-abs offset (cue env)))))))



Creating the envelope:

Code: Select all
    (setf *scratch*
      (snd-copy  (snd-avg (mono *track*)  step step op)))
    ; return *track* to prevent audio from being released from *scratch*.
    *track*) ;*track*)

Looks like I corrected an error here. Anything after a semi-colon is treated as a "comment" and is ignored by Nyquist, so ignore that final ;*track*)
Code: Select all
    (setf *scratch*
      (snd-copy  (snd-avg (mono *track*)  step step op)))
    ; return *track* to prevent audio from being released from *scratch*.
    *track*)


We are setting *scratch* to have the value of (snd-copy (snd-avg (mono *track*) step step op)))
In Nyquist, a "value" may be a number, or a character, or a "string" (text), or a "sound" or one of several other data types. In this case, the value of *scratch* will be a sound (though one with a very low sample rate).

*scratch* should retain its value, but due to memory management in Nyquist, if *scratch* is given a "sound" value, we have to reference that sound when Nyquist returns at the end of the track, otherwise the sound is deleted. With other data types you don't need to do that.

We use SND-COPY to prevent the track from being damaged by SND-AVG.

If, for example, "op" = 2 (the same as "op-peak"), then (snd-avg (mono *track*) step step op) first converts *track* (the track audio) to mono, then finds the peak value of the first "step" samples, saves that to *scratch*, the moves forward by "step" samples and finds the peak level of the next "step" samples, and so on for the rest of the track. *scratch* thus ends up as a "sound" with one sample value for each "step" samples of the original track. In other words, it follows the peaks of the original track.

When we multiply the second track by *scratch*, because *scratch* has a very low sample rate, it is still the same length as the original selection, even though it has far less samples. Where the sound of *scratch* has a value of 1 (0 dB), the second track is amplified (multiplied) by 1, so clearly it remains at its original level. Where *scratch* has a value of 0.5, the second track is amplified by 0.5 (half its original level). Thus the second track as amplified by an amount that follows the contour of the first track.

Does that help?
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)
steve
Site Admin
 
Posts: 45083
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: How to capture envelope from one track and apply to anot

Permanent link to this post Posted by MisterHSP » Fri Mar 18, 2016 12:05 am

steve wrote:When we multiply the second track by *scratch*, because *scratch* has a very low sample rate, it is still the same length as the original selection, even though it has far less samples. Where the sound of *scratch* has a value of 1 (0 dB), the second track is amplified (multiplied) by 1, so clearly it remains at its original level. Where *scratch* has a value of 0.5, the second track is amplified by 0.5 (half its original level). Thus the second track as amplified by an amount that follows the contour of the first track.

Does that help?


It helps. It confirms that I had pretty well puzzled out the functions and results they returned. Thank you.

What has me still puzzled is that when the original selection has a value of 1 (0 dB), the second track is not remaining at it's original level. It is being reduced a bit (approximately 1/3).

In my experiment, I even rendered the original selection from stereo to mono (to eliminate the variances between channels), then duplicated it. The second (duplicated) track was then set to follow the first. My expectation was that they would be the same, still (or at least roughly the same, since the sample rate for the control envelope is much smaller.) What I get is a track that follows the first track but at about .7 amplitude of the original for the passages that are at or near 0 dB. But, it is even lower amplitude on the quieter passages (maybe 2/3).

Is this just a function of the functions or is there a hidden variable in the functions being called?

John (MisterHSP)
MisterHSP
 
Posts: 8
Joined: Wed Mar 16, 2016 9:07 pm
Operating System: Windows 7

Next

Return to Nyquist



Who is online

Users browsing this forum: No registered users and 3 guests