This is not an Audacity problem - it does not change the signal it receives.
If it’s a laptop, try it on battery power (but ensure you set Windows Power Options to warn you before the computer gets to critical power levels).
You can try a different USB cable.
Keep all cables away from mains cables.
Try winding the USB cable a few times through a ferrite ring or adding ferrite beads around the outside of the cable close to both ends:
http://www.maplin.co.uk/p/ferrite-ring-hem3016-qt26d
http://www.radioshack.com/product/index.jsp?productId=2103222 .
Or try a double-shielded USB cable:
double-shielded USB cable - Google Zoeken .
Gale