That’s our joke. Somebody will hit that one song that works absolutely correctly and posts a YouTube Instructional Video how to do it. If you use the exact same song they did, yours will come out perfect, too. By the way, that step-by-step split, invert and add thing in the YouTube videos is already baked into the Vocal Removal tool. You don’t need the extra steps.
There’s a lot of songs that fail. It’s not unusual. If the lead vocal has stereo echo added to their voice, the voice will go and leave a ghost echo behind.
As you said, if the voice is not in the exact left-right center of the stereo sound, it will just get missed altogether. The trick doesn’t work at all with a mono, not stereo song. If you started with a stereo MP3 download, you’re likely to be left with bubbly MP3 compression trash.
There are two tools. If you’re on Audacity 2.1.2, there is also Effect > Vocal Reduction and Isolation… You might try that.