3D Spectrogram

Hi,
I have been using Audacity for a few days now and I am really impressed by it. Easy to use and the results are very impressive. Kudos to the developers!

Anyhoos, I created a bit of 3D artwork using the spectrogram view to help visualize a robin’s song. Link and viewing instructions are here:-

http://starrydave.com/?p=738

Enjoy
Dave

A free 3D spectrogram plugin called SoundRider works in Audacity (Windows only) …
Polygon Audio's SoundRider plugin.gif

I’d be interested to hear how you did that. Would you like to tell us?

Hi Steve,
It’s a WebGL heightmap using the Babylon.js framework. I designed it so that people could generate their own versions just by taking a couple of Audacity screenshots. Let me knock out a quick tutorial and I will post the link here.
Would it be OK to link back here for any comments from the tutorial? (my blog gets swamped by spam)
Dave

On the Mac, you could use the free version of Sugaraudio’s Oscarizor. The free version can only show one channel. Sufficient for most uses, as it is a fairly CPU intensive plugin.

Have just uploaded a tutorial to

http://starrydave.com/?p=744

Maybe someone could test it out with their own sound file.

Cool :slight_smile:

If you want to keep it all open source, you could use Gimp rather than Paint.Net.
And for completeness, a link to the babylon.JS website: https://www.babylonjs.com/

@dwsubc, Are you able to build C++ applications (such as Audacity) from their source code? If you are, there’s some new experimental scripting features in Audacity that may be of interest (if not, you’ll have to wait until they are ready for release :wink:)

Good points. I can’t believe I forgot to reference babylon.js. I have now.

My next step is to try a local build. I switched from C++ to C# a while ago but I could probably still get by. First task will be to add a hide toolbars menu item :smiley: I then want to try directly converting the spectrogram data into HTML5 3D without the screenshot fudge above. Unless that is already covered by one of your experiments?

The “experimental” feature that I was thinking of is the new “scripting interface”. To use this, the optional module “mod-script-pipe” is required, which must be built on the same day as the Audacity application. There are some potential security issues if running mod-script-pipe on a networked computer, but should be safe on a standalone or adequately firewalled machine. The scripting interface is already being used in-house for automating screen captures for the Audacity manual.

Some links about the scripting interface:
https://alphamanual.audacityteam.org/man/Scripting
https://alphamanual.audacityteam.org/man/Automation_Reference

Hey,

I am working with Audacity for some weeks and i saw your post about this SoundRider plugin! It is exactly what i need for my project but unfortunately after installing the plugin i can’t find it in Effects.
May be you can give me some tipps how you install it?

Thanks!

You may have to install Microsoft “DirectX” to make SoundRider work, (see “requirements”)
(I did, but I’m on Vista, which is old).

NB: Audacity only accepts 32-bit plugins, even if your OS is 64-bit.

Hi,

thank you, this looks great! Unfortunately it’s not working for me, neither with your files nor with mine. When I open Birds 03 Spectrogram I just get the audio, but no image at all. With my own files I only get a thin yellow line, which I can move around, but nothing else.

Any suggestions what might be wrong? Something to do with bablyon.js maybe?

Cheers,
phaeton

Same here. I guess dwsubc has removed the visuals, or it has broken, or it requires an older web browser. Either way, you would need to ask the website owner as we have no association with that site.

So, I just looked at the web console in Firefox and I get the following error when I open http://starrydave.com/3d/birds03/birds03.html

Error: WebGL warning: <SetDimensions>: Failed to create WebGL context: WebGL creation failed:
babylon.js:1:134729
* WebGL 2 requires support for the following features: 
  transform_feedback2

In Chrome the console doesn’t show any errors, but it’s still not working.

We can’t help you with this. The problem is between the website and the browser - it has nothing to do with Audacity. If you are bothered about it, I’d suggest that you contact the website owner - we can do nothing about this problem.

Thank you, Steve. I understand that. I was hoping to reach out to dwsubc this way, because I could find any contact information on his website. But you’ve already helped me by confirming that it’s not just me! :slight_smile:

I haven’t looked at this for a while. Let me check on it and I will post below when I get it working.

Thanks dwsubc, and nice to see you back again after all this time :wink:

Thanks Steve, too many hobbies and not enough time :smiley:

It looks like WebGL has gone from v1 to v2 and this is causing the problem. I will get it fixed because I would like to see other users’ examples.

btw, Am currently working on some novel bird videos that will use an Audacity soundtrack and will post details here when finished.

Ok, so I found the problem. Not sure why, but in the “spectrogram.html” file you have to adjust the minimum and maximum height for the mesh by changing:

var ground = BABYLON.Mesh.CreateGroundFromHeightMap("ground", heightmap, 500, 264, 1000, 40, -40, scene, false);

to

var ground = BABYLON.Mesh.CreateGroundFromHeightMap("ground", heightmap, 500, 264, 1000, 0, 10, scene, false);

Other values also work, as long as the second value is higher than the first.

Also you’ll have to invert the “colors” in the grayscale image (“mono.png”) in GIMP (or whatever you’re using), since the brighter a pixel is the higher it is placed on the mesh… Not sure if it would be possible to avoid this by changing something in the code.

So, thanks again for this great idea, dwsubc! Works great for me now.