Regular linear amplification (like turning-up the analog volume control) boosts EVERYTHIHNG including the background noise. In some cases it makes noise audible when it wasn’t previously audible, or simply more noticeable or more annoying.
…Some people think a more sensitive microphone picks-up more room noise, and it does. But it also picks-up more signal so it’s no different than using the less-sensitive mic and amplifying later.
When I boosted it I could hear my guitar better but I could also hear this noise.
You didn’t say if it’s electric or acoustic. Acoustic guitar is tough because it’s “dynamic” (very quiet and rather loud). If you don’t have a soundproof studio you’ll pick-up room noise that isn’t drowned-out when the guitar is quiet or silent. Getting the mic close to the guitar gives a stronger signal-to-noise ratio.
The preamp (built into the interface) also adds some noise (hum or hiss) and it can sometimes be audible with little or no input and the gain cranked-up. A more sensitive mic helps with THAT noise.
Electric guitars are just noisy due to their design. If you play electric guitar, you probably know about humbucking pickpus which can reduce noise. An active pickup puts-out a stronger signal, and it has lower impedance and these things help with noise pick-up further-down the chain, after the pickup itself.
Electric guitars are also dynamic, and you’ve usually audible got noise if you try to play quietly or as a note fades-out. Then if you use a compression-distortion effect, or just crank-up the amp 'till it distorts, you are boosting the background noise while limiting-compressing the loud parts and that makes the signal-to-noise ratio WORSE!
Some guitar players use a noise gate, which can be a “pedal” or Audacity has a noise gate effect. Or you can use Audacity’s noise reduction. These are usually the “last steps” after you’ve done everything else to reduce the acoustic & analog noise.
A noise gate kills (or reduces) the sound when it drops below a threshold. The idea is to kill the sound when there is only noise, but sometimes it will kill the sound as a note is fading-out and if you can hear the background noise cutting in-and-out that can be distracting and worse than just leaving the noise.
Noise reduction tries to analyze the noise and separate it from the signal. But if the noise is bad, you can get side effects and again, 'The cure can be worse than the disease."
Solo recordings are just tough… When you have a “dense mix” of multiple instruments and vocals the noise tends to get drowned-out. This is especially helpful just the guitar is noisy and the other tracks are cleaner. A dense mix will also help to hid the side effects of noise gating or noise reduction, as long as the effect is applied to the guitar track before mixing.