The Red Book standard for Audio Compact Disks call for an automatic 2 second pause between songs or cuts. Your burner or audio authoring software puts that in usually whether you want it there or not. The higher end authoring programs include a check box and a data window where you can produce whatever gap you want including zero.
Audacity doesn’t burn anything, so yes, you need to dig in Windows Media.
If you are trying to burn audio CDs with Windows Media, then I’d stop right now, and find a decent CD authoring package to use - it may well not have a gapless option at all.
For Windows users, I recommend Exact Audio Copy. It’s a bit more complicated, but it’s free and quite powerful. You can also use it to make CUE sheets so you don’t even need to split up the album you’re trying to burn. You can export a big wav file from Audacity and use the CUE sheet function to separate it into different tracks while burning the disc.