Note recognition to gamify learning?

Programming and Development Nyquist
1

Hi - new here, and I’ve searched the forum but may not know the right terminology, so sorry if this has been covered elsewhere…

My two younger kids (8 & 11) respond really well to “gamification” in various topics (the “Prodigy” math game has been huge hit, and really moved their math skills up a few notches).

They are learning to play mandolin and fiddle, and need more practise time - but I don’t want to push, I’ve had too many friends who are technically great players, but hate it because their parents pushed them too hard, and never actually play any more. If there were a “Prodigy”-type game for music, with note recognition via the microphone (or I’d be willing to buy a pickup) that would be ideal, but I have not seen anything like it out there.

If someone knows of one, please post, but otherwise I am thinking that even just a basic call-and-response setup that plays you some notes (and maybe shows the sheet music), listens to you play them back, and then gives you a “hooray” and a score and tracks your total over time would trigger their competitive side and hook them on practising without it being a chore.

Picking single notes (not working on chording, mostly Irish folk melodies are what they are excited about right now) out of a relatively clean background seems like it wouldn’t be a hugely difficult problem, and there’s a ton of stuff out there in abc format to use as input material. Scoring could be something simple based on “here’s a bar, did you get the notes right? did you get the order right? did you get the timing right? now try two bars…” etc.

Could you do that with Nyquist / Audacity? Has it already been done? I haven’t tried Nyquist before but Python seems to handle FFTs really quickly, so I am guessing modern hardware can keep up with the necessary number crunching to do it in real time?

thanks

Mark

2

I don’t think that Audacity will help with your idea.

Accurate note detection is difficult - it’s not just a matter of FFT analysis, because natural sounds (real life instruments) are rich in harmonics, and it is not always the highest amplitude harmonic that determines the pitch. Also, FFT has very coarse frequency resolution at low frequencies unless analysing very large FFT windows (which is slow).

There are a couple of note detection plug-ins for Audacity, but Audacity is not designed for real-time processing. All processing is done “off-line” after the recording is stopped.

As a violinist myself, one thing that encouraged me to practice was playing with others (playing duets / orchestral).

3

This is a little different than what you’re looking for, but if you want to modify sheet music you already have (maybe you have some music for songs they know from Disney or the radio, but it’s a little too hard for them), then you can speed up the process by using a scanner (I like scanscore: https://scan-score.com/en/ [moderator note: scanscore cost from $39 and is available for Windows 10 only] ) that allows you to edit the notes, then convert to pdf to print for them.