Every time I open up a file with Audacity, around 1MB of storage is used up on my hard drive, Do you know how to deal with this?
Working with high quality audio (or video) does require a lot of disk space. There’s no way around that.
Audacity works in very high quality “32-bit float” format.
At the default sample rate of 44100 Hz, a stereo track requires around 20 MB per minute of audio.
Audacity also supports unlimited “Undo” levels, so each time you process a track, the previous state is (temporarily) saved.
On closing Audacity, the Undo data is discarded.
If you’re used to MP3 or other compressed files, that can seem weird, but you can’t edit or change an MP3 without decompressing it into it’s original “real” size and that can be a very nasty surprise.
Even better, if you do edit or change an MP3, you can’t make a new one without the sound quality going down.
Never do production in MP3.
Koz
So how do I remove that space?
Is it a Temporary folder or something?
Check [u]Directories[/u].
Every time I open up a file with Audacity, around 1MB of storage is used up on my hard drive,
If that’s a problem maybe you need a bigger hard drive. 
1MB is a lot of data, but audio (and video) need lots of data so 1MB is small for audio.
A “full” audio CD is about 700MB and if you “rip” and copy those files into Audacity they are converted from 16-bit integer to 32-bit floating-point so that’s twice as much data. You probably won’t use floating-point for your final-format, so that’s usually just temporary, but audio processing “works better” in floating point and virtually all audio editors/DAWs use floating point. ('Some DAWs use 64-bit floating point.)