I have the 2.0.3 version that came with an EZCAP Super USB Cassette Capture device. I successfully transferred about four cassette recordings into MP3 files in my computer. Suddenly, when I click on record in Audacity (before I push play on my device), I get this horrible undulating ringing or feedback. I reinstalled the program, and then it worked OK with one more transfer. However, when I went to do the next one, the loud ringing sound started up again.
The current Audacity version is 2.1.0 from http://audacityteam.org/download/windows.
I would try removing and reconnecting the USB cable, making sure that it’s tight at both ends. If you have another USB cable, try that instead.
It’s unlikely to help, but if you can manage without hearing the tape while you record, try turning off Transport > Software Playthrough at the top of Audacity.