How to dynamically mod volume of one track to match another?

Using Nyquist scripts in Audacity.
Post and download new plug-ins.

If you require help using Audacity, please post on the forum board relevant to your operating system:
Windows
Mac OS X
GNU/Linux and Unix-like

Re: How to dynamically mod volume of one track to match anot

Permanent link to this post Posted by steve » Tue Apr 13, 2010 5:45 am

The method described by Trebor was also using Nyquist functions, just that one of the steps had already been made into a plug-in.
Either way, it will be necessary to process each half of the stereo track separately, then stick the two halves back together when we've finished.

There's going to be a few steps involved, but it's not too difficult, we'll just approach it a bit at a time. When we've finished we could even look at building the effect into a plug-in if you want.

It will be best to practice this on some short practice bits first, you will probably need to do a bit of "tweaking" to get the effect that you want, so if you understand what is happening at each step then it will be much easier for you to adjust it just the way you want.

This will be easiest if both the "tune" and the "control" track have the same sample rate. What sample rate have you been using? (your current project sample rate will be indicated in the lower left corner of Audacity)

First let's split a stereo track into 2 mono tracks:

Record or import a short stereo track.
Click on the track name and from the drop down menu select "Split stereo to Mono"
You now have 2 mono track.
Click on the name of the upper track and select "Name", then change the track name to "Left"
Do the same with the other track and call it "Right"

Select the left track and export it using "Export Selection" and call the exported "test-left.wav"
Do the same with the other track and call it "test-right.wav"

Close and restart Audacity, then import the two files "test-left" and "test-right".

Has that all worked correctly?
Any questions so far?

For the next step I need to know:
Do you want the left channel of your tune to follow the left channel of the control track, and the right to follow the right,
or do you want both tracks to follow an average volume of the control track (left+right)
or some other combination?
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)
steve
Site Admin
 
Posts: 47011
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: How to dynamically mod volume of one track to match anot

Permanent link to this post Posted by Raiten » Tue Apr 13, 2010 6:08 am

Hi Steve,

Great, I have those steps completed. Both tracks have a sample rate of 44100Hz, 32-bit float.

I would like the left channel of the tune to follow the left channel of the control track, same deal on the right.

I really appreciate you taking the time to walk me through this.

Thank you,
Matt
Raiten
 
Posts: 9
Joined: Mon Oct 19, 2009 11:20 pm
Operating System: Please select

Re: How to dynamically mod volume of one track to match anot

Permanent link to this post Posted by steve » Tue Apr 13, 2010 7:15 am

Good so far :)

Now this is the "snd-follow" bit that will turn your control track into a volume envelope -

Don't use this on a long track - it can be very slow and could even crash Audacity if the track is too long.
Up to 30 second should be fine.

We will modify this before we use it for real so that it can handle longer tracks, but so you can start playing with it now, let's have a look at the command:

Code: Select all
(snd-follow sound floor risetime falltime lookahead)


There is a full description of it here: http://www.audacity-forum.de/download/e ... l#index517

The function name is "snd-follow" and everything after that are the function parameters - the (link above) description will give you an idea of what each of them does.

In Nyquist, as in the LISP programming language, all functions are written within parentheses (brackets).

To use this function we will enter it using the "Nyquist Prompt" effect in the Effects menu.

This next bit will only work on a mono track.

1) Import or record a short section of audio - if it is in stereo, split to mono and discard one of the tracks.
2) Ensure that the track, or part of it is selected and from the Effects menu select "Nyquist Prompt"
3) Copy this bit of code and paste it into the Nyquist Prompt text box.
4) Click the "Debug" button - we could just click "OK", but the debug button gives us the opportunity to see any error messages or debug information if anything goes wrong.
The result of this will be a very low frequency waveform that rises and falls, following the "shape" of the original waveform. If you try to play this new track you won't hear anything much, the main frequency is far too low, but we don't want it for sound, we want it as a control signal that we can then use on the "tune" track.

Here's the code:
Code: Select all
(let
   ((floor 0.001)
   (risetime 0.1)
   (falltime 0.25)
   (lookahead 200))
(snd-follow s floor risetime falltime lookahead))


Explanation:
The last line is the "snd-follow" command.
You will notice that "sound" has been replaced by "s".

"s" is a special variable that is used by Audacity to pass sound from the track being processed to Nyquist (Nyquist being a separate program that has been embedded into Audacity).

There are then the variables floor, risetime, falltime and lookahead. We did not need to use these names, we could have just called them a, b, c and d, but using these longer names keeps it clear what each variable does.

Before these variables can be used, they must each be given a value - there are several ways that we could have done this, but a simple and tidy method is to use the "let" function.
"LET" is listed in the Nyquist manual here: http://www.audacity-forum.de/download/e ... #index1183
but it is exactly the same as in XLISP (as are many Nyquist functions) and there is a more comprehensive description in the XLISP manual here: http://www.audacity-forum.de/download/e ... ef-148.htm

The "bindings" in the "let" function are the part where we allocate values to our variables. In this case our variables are floor, risetime, falltime and lookahead. In the above code we have allocated (bound) the values 0.001, 0.1, 0.25 and 200 to each of these variables respectively.

Note that the rise-time and fall-time are in seconds, but the lookahead value is in samples.



Have a play with this, try different values and different and see how they work. If you manage to crash Audacity just restart it. If you totally freeze Audacity, reboot. You can't actually break anything with this.

I've got to get off now, so have a good play and let me know how you get on.
Catch you later.
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)
steve
Site Admin
 
Posts: 47011
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: How to dynamically mod volume of one track to match anot

Permanent link to this post Posted by steve » Tue Apr 13, 2010 7:18 am

Forgot to ask - roughly how long is the actual track that you want to process?
Any idea what the spec, or age of your computer is? how much RAM?
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)
steve
Site Admin
 
Posts: 47011
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: How to dynamically mod volume of one track to match anot

Permanent link to this post Posted by Raiten » Tue Apr 13, 2010 4:45 pm

Hi Steve,

Following right along, I now have a 30 second inaudible control signal from the left channel of my control track. I would like the finished track to be 45 mins to 1 hour long. I am working off a Macbook Pro (2.5Ghz Core 2 Duo, 4GB RAM), but I also have a tower that I built for 3D graphics that I could jump on if needed (3.0Ghz Core 2 Quad, 8GB RAM).

Thanks,
Matt
Raiten
 
Posts: 9
Joined: Mon Oct 19, 2009 11:20 pm
Operating System: Please select

Re: How to dynamically mod volume of one track to match anot

Permanent link to this post Posted by steve » Tue Apr 13, 2010 10:12 pm

The Mac will be fine, just checking that your not stuck on an antique. As for the other one - I could take it off your hands if you like.. say, $50 ? or swap it for my lovely Pentium III 500MHz :D

OK enough of the frivolities, I'm sure you're itching to get on with this.

Remember I was saying that we were going about making the control signal in a rather inefficient manner - let's make it faster and more efficient.
The wave that you are making the control signal from is in "high definition" - lots of little detail for producing the audio wave - what's your sample rate 44100Hz? 48000Hz?
(it will say on the left side of the track)

For audio, you need this high sample rate so that you can capture all of the high audio frequencies, but for our control signal all we need is to follow the general "shape" of the sound. We can easily manage with a sample rate 100 times lower than is required for the sound. This will reduce the number of samples that (snd-follow..) has to process by 10000% - a pretty significant saving.

There's a couple of ways that we can do this - probably the most obvious is to just resample the sound, but there's another method that is really neat, and helps to smooth out the control wave even before it gets to (snd-follow)

The function I have in mind is "snd-avg" - you can probably guess what this does - it looks at average values of blocks of samples. For each block of samples it will output 1 sample that represents the either, the "average peak value" of that block, or the "average RMS value" of that block. We'll be using the average peak value.
The function is described here: http://www.audacity-forum.de/download/e ... l#index494

So let's have a go - get your self a mono sound clip as before, and apply this code through the Nyquist Prompt box.

Code: Select all
(let*
   ((blocksize 100)
   (stepsize blocksize))
   (snd-avg s blocksize stepsize OP-PEAK))


Oh no - where's my sound gone?!
Press the "Fit Selection" button or Ctrl+E - this will zoom in on your processed track.
Do you notice that the "shape" of the processed track is the same as the original - just 100 times shorter?

Try this on a few audio clips, and watch the "shape" of the audio.
Try setting "blocksize" to different values.
What happens with a large block size?
What happens with a small block size?

Explanation:
Most of this code is similar to the previous code, but notice that there is an asterisk (star) after "let".
The difference between "let" and "let*" is that without the asterisk the bindings will be made in no particular order, but with the asterisk the binding are made in the order that they appear.

The order is clearly important in this example because "stepsize" cannot be set to "blocksize" until "blocksize" has itself been set.
"let*" is described in the XLISP manual here: http://www.audacity-forum.de/download/e ... ef-149.htm

The final line (snd-avg s blocksize stepsize OP-PEAK)
"snd-avg" is the function name
"s" is the sound passed from Audacity to Nyquist
"blocksize" is how many samples are processed in each block
"stepsize" is how many samples to move on before taking the next block.
"OP-PEAK" says that we are looking at peak values.

Warning! There's an error in the manual - or at least an inconsistency with Audacity's implementation of Nyquist.
The manual says that "stepsize" may be larger or smaller than "blocksize".
IT CAN'T - "stepsize" must be smaller or equal to "blocksize".
If "stepsize" is greater than "blocksize", Audacity is very likely to crash.

In our case, we want to take the average of 100 samples, then step forward 100 samples and take the next 100 sample - this way the function looks at each sample exactly once.

You're probably now thinking "this won't work - I want my control signal to be the original length, not 100x shorter".
Trust me, I'm a musician, it'll work out.

In Audacity, Nyquist has a very peculiar way of dealing with time - I shan't go into it in detail because it gives me a headache thinking about it, but in brief, the "selection" is taken to be "one unit of time". Although our selection shrank 100 fold, as far as Nyquist is concerned it is still "one unit of time" and it will still do what we want it to do.

----------------------------------------

The final piece of the puzzle that we need before we put it all together, is how to deal with stereo tracks.

Firstly, how to make a stereo track out of 2 mono tracks. This bit's easy :)

1) Get 2 mono tracks into Audacity - one track above the other (importing 2 mono tracks will automatically place them one above the other.
2) Click on a blank part of the track information box below the track name - hold the mouse button down and you can drag that track up and down to put the tracks in whichever order you want. Does that work for you?
3) Click on the name of the track at the top, and from the drop down menu click "Make Stereo Track" - Bingo! you have made a stereo track out of two mono tracks.
4) Press Ctrl+Z (undo), swap the tracks round and make it into a stereo track again - you must always select "Make Stereo Track" in the upper track.

In stereo tracks, the upper track is the Left channel and the lower track is the Right channel.

When we were dealing with mono tracks, we used the variable "s" to pass the audio data from Audacity to Nyquist. If we are working with a stereo track, we have 2 sets of data - the Left channel and the Right channel. We can no longer use an ordinary "variable" because variables can only hold one value (in this case, one sound) and now we have 2.

What we use instead of a variable is an "array". An array is like a box into which you can put several values. For a stereo track we will be using a box that has 2 compartments - we'll put the left channel into the first compartment and the right channel into the second compartment.

We don't actually have to worry about how to create this box and put the left and right channels into it - Audacity does that for us. This array has a very easy to remember name - it is "s" (just the same as for mono sounds, except that "s" is now an array and not just an ordinary variable).

The thing that we do have to concern ourselves with, is how we get each element (sound) out of the array. For this we use the cryptically named function "aref"
http://www.audacity-forum.de/download/e ... ef-029.htm

So, for a stereo track we have an array called "s" which has two elements - the left channel and the right channel. As is typical with computers, the elements are counted starting at 0.

So the first element of the array "s" (the left channel) can be accessed with:
(aref s 0)

The second element of the array "s" (the right channel) is accessed with:
(aref s 1)

there is a special name for an array that has 2 elements, and that is "vector". This is also the name of the function that we use to put the elements back together again.
In order to send a stereo sound back from Nyquist to Audacity, we just create a "vector" with two sounds: (vector sound-left sound-right)

Let's try a practical example:
1) Get two mono sounds into Audacity - make sure that they look and sound very different.
2) Make the two mono tracks into one stereo track - you should be able to easily see that the left and right channels are different from each other.
3) Select the track and apply this code using the Nyquist prompt.
Code: Select all
(vector
  (aref s 1)
  (aref s 0))


What this does is to create a vector where the first element is (aref s 1) and the second element is (aref s 0)
We've already established that (aref s 1) is the Right audio channel and (aref s 0) is the Left audio channel, but the vector that we are sending back to Audacity has (aref s 1) as the first element and (aref s 0) as the second element - we have swapped the left and the right channels.

-------------------------------

Well we're almost there now - some things for you to play with while I rest my weary typing fingers.
Let me know how you get on and next we'll look at putting all of this together.
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)
steve
Site Admin
 
Posts: 47011
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: How to dynamically mod volume of one track to match anot

Permanent link to this post Posted by Raiten » Wed Apr 14, 2010 5:06 pm

Steve, you're awesome. I really appreciate you taking the time to not only walk me through this, but also teaching me how this process works. The week gets busy for me starting today and I won't have time to work on this next step until Monday. Thank you for being so helpful.

Matt
Raiten
 
Posts: 9
Joined: Mon Oct 19, 2009 11:20 pm
Operating System: Please select

Re: How to dynamically mod volume of one track to match anot

Permanent link to this post Posted by steve » Wed Apr 14, 2010 6:05 pm

No problem, I'm finding it interesting.
Post back when you're ready for the next instalment ;)

If there's bits that I've not explained clearly enough, just say - I'm trying to pace it so that you get your usable effect in a reasonable time frame, but in enough detail so that you understand what and how it's happening so that you are able to tweak it to suit your needs.
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)
steve
Site Admin
 
Posts: 47011
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: How to dynamically mod volume of one track to match anot

Permanent link to this post Posted by Raiten » Mon Apr 19, 2010 2:30 am

Steve, I have been playing with the last set of instructions you gave me. My original track is 44100Hz.

-snd-avg-

A smaller blocksize produces a longer result with the snd-avg function and a larger blocksize produces a shorter result. That part makes sense. The larger blocksize also produces a result with a larger amplitude. Why is the amplitude larger?

-making a stereo track-

Holding the mouse button down works for me to drag and change the order of the tracks and I successfully created a stereo track from two mono tracks.

-aref-

All set on the aref and I see how that function swapped the left and right channels.

What's next teach? :D

Matt
Raiten
 
Posts: 9
Joined: Mon Oct 19, 2009 11:20 pm
Operating System: Please select

Re: How to dynamically mod volume of one track to match anot

Permanent link to this post Posted by steve » Wed Apr 21, 2010 10:53 pm

Apologies for the delay Matt, it's been busy at work.

Raiten wrote:The larger blocksize also produces a result with a larger amplitude. Why is the amplitude larger?

That's because it is tracking peaks, so with a larger block size, each block of samples is likely to catch a higher peak than a smaller block size.
My previous description of (snd-avg) was a bit misleading - since you raise the question I'll try to be more precise:

From the manual:
(snd-avg sound blocksize stepsize operation)
Computes the averages or peak values of blocks of samples. Each output sample is an average or peak of blocksize (a fixnum) adjacent samples from the input sound. After each average or peak is taken, the input is advanced by stepsize,


So in a sense this function can do one of two different operations:
It can either, create a sample based on the average value of the samples in the block (when using "OP-AVERAGE")
or it can create a sample based on the peak value of the samples in the block (when using "OP-PEAK")

In this latter case it is not an "average" of anything - the sample value that is produced by looking at each block is determined by the sample that has the highest absolute value. The "absolute" value is the magnitude regardless of +/- sign, so OP-PEAK will always output positive samples.

In the case of "OP-AVERAGE" it IS important if sample values are positive or negative. The output sample value is equivalent to the sum of each sample divided by the number of samples. For normal waveforms and a large block size, the average will be close to zero, which is not useful for us with this application.

-----------------------------------

We're still not quite ready to process anything big yet, so practice these on short snippets of sound.

Preparing our sounds.

It should be no surprise that we are going to want mono sounds, but that's OK, we've covered how to split stereo sounds into mono.

The plan is, that we split our "tune" track into two mono tracks, then combine each of these with our mono "control track" so that we have 2 stereo tracks.
Each stereo track will contain the "tune" in one channel, and the "control" in the other channel.
We can then "apply" the control to the tune - Remember that Audacity Nyquist can only process one track at a time, which is why we need to combine the control and the tune in the same (stereo) track.
When we have processed both channels of the tune, we will have 2 stereo tracks.
We will then convert each of these stereo tracks to mono and recombine them as our stereo result.

Before we do that, there's a couple of things that can help the effect.

Preparing the Tune track:

I assume that your "tune" track already has "dynamics" (variations in volume). We will probably get a better effect if we even out these dynamics so that the effect is being applied to a sound with a fairly consistent amplitude.
A very good way to do this is to use a plug-in called "Chris's Dynamic Compressor", but rather than get too distracted, for now we'll just use the standard Audacity compressor effect.

To compress a track (we will go quite extreme with this)
1) Select the track
2) Effect menu > Amplify > OK (just to bring our peak level up to 0dB before we start)
3) Effect menu > Compressor
4) Set:
Threshold = -30
Noise Floor = -40
Ratio = 10:1
Attack and decay on minimum
Make up gain = selected
Compressed based on peaks = selected

That should give you a track that is all pretty loud.
Use the track drop down menu to "Split to Mono".
Use the track drop down menu to change the name - call them "Tune L" and "Tune R" (or something like that).


Preparing the control track:
Nothing much special here - we need it to be mono, so you can use "Tracks menu > Stereo track to mono" (if it was a stereo track).
We also want to take maximum advantage of the dynamics, so use the Amplify effect (with default settings) to bring the amplitude up to 0dB.
We will need a second copy of this track, so select the track then press Ctrl+D (this will duplicate the track).
Name these tracks "Control" or something similar.

Combining tracks:
Move the tracks so that you have a "tune" track above a "control" track, then combine them so that you have the tune in the left (upper) channel and the control in the right (lower) channel.
We will be able to access the tune channel with (aref s 0) and the control with (aref s 1).

----------------------------------

Combining our Nyquist functions.

There are a number of ways that we could do this, but I think this method will be easiest to follow.
We can "set" a variable to a given value using the function (setf ...)
This is a very versatile function, but I'll not go into great detail as it can get a bit complicated - instead we'll look at some simple examples:

Code: Select all
(setf my-variable 42)
(print my-variable)


This has simply set a variable, which we have given the name "my-variable" to the value 42
The (print) function prints out the value of the variable.
It doesn't do anything to the sound in the track, it just prints out the result as a screen message.

Let's try something with a mono sound;
Code: Select all
(setf half 0.5)
(setf mysound s)
(mult mysound half)


Here we set a variable "half" to the value 0.5
Then we set a variable "mysound" to the sound in the track, which as said before is held in a special variable "s".
Not surprisingly the function (mult) multiplies things - in this case it multiplies the sample values in "mysound" by the value of "half". In other words, the sound will be scaled to half of its original amplitude.

-----------------------------------------

Let's consider this bit of code:
Code: Select all
(setf newsound
  (let*
    ((blocksize 1000)
    (stepsize blocksize))
    (snd-avg mysound blocksize stepsize OP-PEAK)))


This is almost identical to something we used earlier.
We are setting "newsound" to be the result of (snd-avg mysound ....)

This gives us a nice easy method of passing the result of one function into another function (it's not necessarily the best way, but it's simple).

So, let's use (snd-avg ... OP-PEAK) to create a low frequency version of the control sound and then put that into our (snd-follow ....) function.

(I've used upper case letters for the variables just to make them easier to see - Nyquist is not case sensitive so it doesn't matter if we use upper case or lower case).

Code: Select all
(setf R-CHAN (aref s 1))

(setf LOW-R-SOUND
  (let*
    ((blocksize 1000)
    (stepsize blocksize))
    (snd-avg R-CHAN blocksize stepsize OP-PEAK)))

(setf ENVELOPE
  (let
    ((floor 0.001)
    (risetime 0.1)
    (falltime 0.2)
    (lookahead 2))
  (snd-follow LOW-R-SOUND floor risetime falltime lookahead)))


I'll only cover this briefly now because it's getting late - we can discuss this more later if you have questions.

We are setting "R-CHAN" to the sound in the right channel of our stereo track.
Then we use that in (snd-avg ...) to create a low-sample-rate input for snd-follow.

We then want to "apply" that to the "tune" channel, which we do by "multiplying" it:

Code: Select all
(mult (aref s 0) ENVELOPE)


So putting the whole lot together:

Code: Select all
(setf R-CHAN (aref s 1))

(setf LOW-R-SOUND
  (let*
    ((blocksize 1000)
    (stepsize blocksize))
    (snd-avg R-CHAN blocksize stepsize OP-PEAK)))

(setf ENVELOPE
  (let
    ((floor 0.001)
    (risetime 0.1)
    (falltime 0.2)
    (lookahead 2))
  (snd-follow LOW-R-SOUND floor risetime falltime lookahead)))

(mult (aref s 0) ENVELOPE)


-----------------------------------------

Well I think that's all the bits that you need, though a bit rushed at the end.
Have a go and see how you get on.

Note that Nyquist in Audacity is not particularly good for long tracks, eventually you will run out of RAM and Audacity will crash, so practice on short samples, then work your way up in size once you have it working.

speech-modulating-music-dynamics.mp3
(275.58 KiB) Downloaded 439 times
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)
steve
Site Admin
 
Posts: 47011
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Previous

Return to Nyquist



Who is online

Users browsing this forum: No registered users and 1 guest