I know this question is a bit old, but it turns up in google well so maybe it will help some others.
I noticed that the USB cable included with the Samson is very thin, long, and feels cheap. I tested with several other USB cables and found one which eliminated the irritating high frequency sound. This USB cable has a "Ferrite bead", and is much shorter than the cable in the package. I included a photo.
"A ferrite bead or ferrite choke is a passive electric component that suppresses high frequency noise in electronic circuits. It is a specific type of electronic choke. Ferrite beads employ high frequency current dissipation in a ferrite ceramic to build high frequency noise suppression devices. Ferrite beads may also be called blocks, cores, rings, EMI filters, or chokes."
There is some small noise left, but not much. I just bought Samson Sound Deck for $ 4 which reduces the remaining noise almost completely.
In my setup I also noticed that the power supply of my macbook makes a lot of high frequency noise in my headphones. After the power supply included with my macbook broke I bought a not original one. It was only $ 25, while the original one was $ 80. I bought this thing a year ago, never noticed any problems till I started to record audio.