Nyquist Drum Replacer

Hi Steve,

I have an idea for a plugin that will be useful to me, and probably others too.
However, before I embark on this coding journey, just wanted your opinion as to how do-able it would be.

Let’s say we have a track, on which there are only drums, however I want to change the drum beat (sound not tempo)
with another.

This other (new) drum beat/sound can be from a file, or a single sound (or selection) from another track, or even a selection
at the beginning of the same track that will be modified, which ever is easiest.

Now the question is, how easy/difficult would it be to code some Xlisp to detect the drum beats and replace them with the new one?

It can be on a new track or by replacing the “old” drum beats on the existing track.

I have looked at existing Nyquist plugins, but nothing that would fit the bill.

Thanks.

In no particular order:

See “Beat Finder” Beat Finder - Audacity Manual
It works relatively well on single drum tracks.

You will see in the code that it uses a low-pass filter (LP sound) but if you test on a drum only track you may find that it works better without.

(defun bass-tracker (sig)
  (let* ((bass (lp sig 50))
         ;(snd-follow sound floor risetime falltime lookahead)
         (follower (snd-follow bass 0.001 0.01 0.1 512)))
    (force-srate 1000 (lp follower 10))))



From a WAV file is pretty easy, and a very flexible solution.

  • You could use a hard coded file path, such as “C:\User\Paul2\Documents\drums\default.wav” (easiest), but that makes it specific to your computer, and platform specific even if the user name is input through the plug-in’s GUI.
  • You could use a text input widget to enter the full path to the WAV file, but that’s awkward to use, especially on macOS.
  • You could use a File Button widget, which is probably the best option for users, but more tricky to code.

Another option could be to synthesize the drum sounds. Nyquist is very powerful for synthesizing sounds, though it can be very difficult to synthesize realistic sounds. The sounds in “Rhythm Track” are all synthesized: https://github.com/audacity/audacity/blob/master/plug-ins/rhythmtrack.ny and “Risset Drum” is easily modified to create a range of synth-drum type sounds: https://github.com/audacity/audacity/blob/master/plug-ins/rissetdrum.ny


Whichever method you use for getting the new drum sound, see SND-ONESHOT for a way to trigger the drum: Nyquist Functions


I would highly recommend that you break down you idea into smaller parts, and write each part as code that you can run and test individually. You will have at least 4 parts:

  1. Beat detection
  2. Importing / capturing synthesizing the new drum sound
  3. Triggering the sound
  4. The graphical interface (don’t start on this until you have some working code for the other parts)

That is fantastic advise and information, thank you very much Steve.
You have given me loads of ideas and an excellent starting point.

Will update here on progress.

Thanks again.

I’ve just noticed in the beat.ny code:

(force-srate 1000 ...

That’s almost pointless imo. The only slightly useful thing that does is that in the final DO loop

(do ((time 0.0 (+ time 0.001))

time can be incremented by a round decimal number.

Better in my opinion, would be to leave out the “force-srate” and calculate the actual sample rate of the sound returned from the BASS-TRACKER function.

(let ((beats (bass-tracker (mix-to-mono *track*))))
  (setf rate (snd-srate beats))
  ...

Then, in the DO loop, you just need to count samples, and you can convert from samples to time in seconds with:
seconds = number-of-samples / sample-rate

Understood.

As you suggested, starting off by using, and looking at the code in the beat detect plugin.
On an isolated drum track, it works very well.
The test track is a “song” at 136 BPM (4/4 timing) and just over 3 minutes long.
There were only 22 errors, either missed beats (probably due to the threshold setting) and some double labels.
As a ratio, that error is very acceptable and quick and easy to fix.
Screen Shot 2021-06-08 at 1.50.17 PM.png
Now, just thinking out loud…wondering if there may be another way to approach this.
Since the timings are already known in the label track, wondering how easy it would be (or complete madness) to then
analyze the label track, create a new audio track and just place the new drum sound at each label start.
Screen Shot 2021-06-08 at 2.18.18 PM.png
Not saying it’s easier, it’s probably not, but just want to consider all options first.

Ok, going with your initial idea Steve.

Had a look at the “beat.ny” code, which is version 1 Xlisp syntax.

Where is the actual new label track being created in the code?

With version 4, it can be done this way:

(aud-do "CursTrackStart:")
(aud-do "AddLabel:")
(aud-do (format nil "SetLabel: Text=~s" tname))

But version 1?

Is it perhaps this line?

(if (and p (> v thres)) (setq l (cons (list c "B") l)))

And more specifically either “cons” and/or “list” ?
Searched for both of these but found no info like parameters they take.
I tried in the prompt:

(cons (list 2 "B") 1) ; just a guess with the vars, assumed where I put the "2", it's a time offset and "1" is the length.

No error, but also no label track created.

See this section on “Return Values”: Missing features - Audacity Support


With reference to the latest version of beat.ny (https://github.com/audacity/audacity/blob/master/plug-ins/beat.ny)

(let ((beats (bass-tracker (mix-to-mono *track*))))
  (setf peak-sig (peak beats ny:all))
  (setf threshold (* threshold peak-sig))
  (do ((time 0.0 (+ time 0.001))
       (val (snd-fetch beats) (snd-fetch beats))
       (flag T)
       labels)
      ((not val) labels)
    (when (and flag (> val threshold))
      (push (list time "B") labels))
    (setf flag (< val threshold))))

A local variable called “labels” is declared in the DO loop. It has not been set to a value, so it has a value “NIL”.
The ‘NIL’ constant represents an empty list or the boolean value “false”. In this case it is used as an empty list.

Each time (“when”) that “flag” is “true” and the value of “val” is greater than the value of “threshold”, a list "(list time “B”) is pushed onto the list “labels”.
Thus, “labels” ends up as a list of lists, in the form:
(list (list t0 “B”) (list t1 “B”) … (list tn “B”))

When that “list of lists” is returned to Audacity, Audacity interprets it as point labels.

“CONS” is described here: XLISP cons

“PUSH” is shorthand (it’s a LISP macro if I recall correctly).
(push val my-list) is equivalent to (setf my-list (cons val my-list))

I know that you didn’t write it, but that code is horrible.
What is that character after “(setq”? Is it the number “1”, a lower case “L”, what does it represent? It is not obvious, and it so easily could be.
What is the “test” in the IF statement? Don’t make me count parentheses.

Much better would be something like

(if (and is-peak (> val thres))
    (setf labels (cons (list t0 "B") labels)))

or better, since there is no “else” clause:

(when (and is-peak (> val thres))
  (setf labels (cons (list t0 "B") labels)))

or

(when (and is-peak (> val thres))
  (push (list t0 "B") labels))

A bit off-topic, but I thought that you might find this interesting.
It is a very simple (too simple to be very useful) implementation of an algorithm to calculate the tempo of a drum track. It works reasonable well with click tracks, but would need to be more sophisticated to work well with a real drum track.

The basic algorithm is well known, and is described here: https://sound.stackexchange.com/questions/27460/how-do-software-algorithms-to-calculate-bpm-usually-work

;nyquist plug-in
;version 4
;type analyze
;name "Guess Click Track Tempo"

(setf tracker-rate 200) ;target sample rate for tracker
(setf lo-bpm 30)
(setf hi-bpm 300)

;; Convert bpm to beats per seconds.
(setf lo-bps (/ lo-bpm 60.0))
(setf hi-bps (/ hi-bpm 60.0))

(defun peak-tracker (sig)
  ;; Returns a low sample rate control that tracks peaks.
  (setf step (round (/ *sound-srate* tracker-rate)))
  (snd-avg sig step step op-peak))


(let ((tracker (peak-tracker *track*))
      best-bps
      (best-peak 0))
  (do ((bps lo-bps (+ bps (/ 60.0))))
      ((> bps (1+ hi-bps)))
    (setf p0 (peak (comb tracker 20 bps) 2000))
    (when (> p0 best-peak)
      (setf best-peak p0)
      (setf best-bps bps)))
  (if (< (round (* 60 best-bps)) hi-bpm)
      (format nil "~a bpm" (* 60 best-bps))
      "BPM not found."))

Wow, fantastic info once again.

As regards the “beat.ny” code, makes perfect sense now, yes I agree that original code is a bit confusing.
Once I read your explanation and your code simplification, together with what is written in the page about return values:

When the return value is a character or string, a dialog window will appear with the data displayed as text.

Then further down…

If an appropriately formatted list is returned to Audacity, a label track will be created below the audio track(s).

Bingo!! The penny dropped.

The format is then “constructed” and “pushed” like: ((number “string”) (number “string”) … )

I also really like your code for the tempo.
Was going to surprise you by adding BPM, but I arrived at it via another way.
If you look at the table of values I posted earlier:
Screen Shot 2021-06-08 at 9.06.56 PM.png
These values are in the “time” variable, so if we take the delta of any two consecutive values, invert
and multiply by 60, we get the BPM.

Taking the first two values:
(sorry, looks a bit convoluted as doing equations on the forum does not format too nicely)

0.744150 - 0.300150 = 0.440
1/0.440 = 2.2727
2.2727 * 60 = 136 BPM

i.e.

1/(t2-t1) * 60

Interesting project, just had my “din-dins” so going to carry on coding.

Right…creating label tracks is not that difficult after all.
The code below…

;type analyze
(setf sel-begin (get '*selection* 'start)) ; start offset
(setf sel-end (get '*selection* 'end))  ; end offset
(setf sel-dur (get-duration 1)) ; duration of selection

(setf end-mark (- sel-end sel-begin))
(setf start-mark (- end-mark sel-dur))

(setf labels NIL)

(push (list start-mark "START") labels)
(push (list end-mark "END") labels)

Produces a “START” and “END” label from a track selection.
Screen Shot 2021-06-08 at 11.11.18 PM.png
The only tricky part was getting to understand how Nyquist handles time.

That’s a very good way to do it, so long as you detect exactly one drum hit for each beat. It becomes rather complicated if the drum is playing some other rhythm, or if the beat detector misses some beats or adds in extra ‘false’ beats.

Yes, if there are false triggers or missing ones, then the BPM will be out.
To overcome this, could do three measurements at different times and take an average.
These measurements could be done at the beginning, middle and towards the end.
The timings will be in a list, which is essentially a formatted array, I’m assuming.

As for the drum sample/beat that will be replacing with, thought about something there as well.
The file option is not great due to the shortcomings of Nyquist.
Also, depending on the BPM of the song, it will have an effect on how long the decay of the drum can be.
So, even if the file option was OK, it would be difficult to visualize the length of the replacement drum sound.
Too long and it will be a mess.

So how about this:

Create a track, on this track put some candidate drum replacement sounds, with some silence between them.
With the original drum track underneath it, it’s now easy to compare them.
We can also preview all the individual candidate drum samples and, once one is chosen, we can manipulate it.
Shorten it, adjust tonal qualities, etc.

Once done, we can use that sample as the replacement sound.

Now, the question is, what is involved in selecting that wanted sound, run a Nyquist plugin, it stores it in an array then
into scratch.
Similar idea to your PUNCH/ROLL plugin.

Now we run the “main” plugin and the sound gets put in a new audio track at the right times.

Do-able?

What’s the problem?
Does this work for you?

;type generate

;control filename "Select a mono WAV file" file "" "*default*/drum.wav" "WAV file|*.wav;*.WAV" "open,exists"

(defun get-sound (fname)
  (let ((fp (open fname)))
    (cond
      (fp (close fp)
          (s-read filename))
      (t  (format nil "Error.~%~s~%cannot be opened."
             fname)))))

(let ((drum-snd (get-sound filename)))
  (if (arrayp drum-snd)
      (format nil "Error.~%~s~%is stereo."
              filename)
      drum-snd))

See here about s-read: Nyquist Functions

A very simple drum machine:

;type generate
;name "Simple Drum Machine"

;control filename "Select a mono WAV file" file "" "*default*/drum.wav" "WAV file|*.wav;*.WAV" "open,exists"
;control number "Number of beats" int "" 8 1 32
;control tempo "Tempo" int "bpm" 120 30 300

(defun get-sound (fname)
  (let ((fp (open fname)))
    (cond
      (fp (close fp)
          (s-read filename))
      (t  (format nil "Error.~%~s~%cannot be opened."
             fname)))))

(defun do-sequence (d-snd)
  (let ((delay (/ 60.0 tempo)))
    (simrep (i number)
      (at (* i delay) (cue d-snd)))))

(let ((drum-snd (get-sound filename)))
  (if (arrayp drum-snd)
      (format nil "Error.~%~s~%is stereo."
              filename)
      (do-sequence drum-snd)))

Some explanation:

Generator plug-ins will create a new mono track if there is no selection. It’s not currently possible for a normal Nyquist plug-in to create a stereo track.
Generate type plug-ins count time in “seconds” (;type process and ;type analyze stretch time so that “1 unit” of time is the length of the selection).

(defun get-sound (fname)
  ;;; Returns the audio file 'fname', else an
  ;;; error message if the file is not readable.
  (let ((fp (open fname)))  ;try opening the specified file
    (cond
      ;; If 'fp' exists (is not NIL), the file exists.
      ;; We have opened the file, so close it before doing
      ;; anything else.
      (fp (close fp)
          (s-read filename))  ;Read the audio file with default options.
      ;; 'Else' return an error message.
      (t  (format nil "Error.~%~s~%cannot be opened."
             fname)))))



;;; Add the sound at specified intervals for the
;;; specified number of times, and return the result.
(defun do-sequence (d-snd)
  (let ((delay (/ 60.0 tempo)))
    (simrep (i number)
      (at (* i delay) (cue d-snd)))))

Thank you Steve.
That works nicely.

Considering how many options have been discussed on this thread, perhaps it’s best not to make one single plugin, but a few of them.
The general idea is to enable the creation of a simple song using only Audacity.

So one track for bass drum, another for hi-hats, cymbals etc.
Once those have been created, can mix them into a percussion stem (if need be), then move on to the synth track, guitar, vocals, etc.

What I have in mind then is:

  • One plugin to detect the drums and just create labels and possibly a BPM count as well.
    This is optional and only needed if one wants to add to an existing song, change it or play along with it.

  • A variation on the above, but it inserts actual samples instead of just labels.
    There is use for both types.

  • Another plugin, your simple drum machine to create the percussion tracks, one at a time.
    A nice addition is to be able to include a pattern, i.e. cymbals only every third beat, etc.
    This can then be used to add different instrument samples.

I like the idea of being able to record myself tapping a rhythm on a table, and then replace the tap sounds with a drum. It would be perfectly sufficient for each recorded track to be one drum only, and has the advantage that I can then mix the tracks as required. I’m not aware of an existing plug-in to do that.

Tapping will work and also, as you know, there is a click track which can be useful in certain situations.

It would be perfectly sufficient for each recorded track to be one drum only

Absolutely, sorry if my explanation was not clear.
What I meant was, using your drum machine to create separate tracks for each instrument.