Audacity is not bad, first off. But saving files sucks. It always uses some odd, outdated folder, and mostly i have to look up, from which special folder the track is coming from. That sucks. Please make an option to use the source folder, from which the track was taken! So i can just overwrite the file, instead of always ending up with the export being written to who knows where.
Have you looked at Preferences > Directories ? Does anything there help?
To be fair to the developers, I remember long discussions on this very topic several years ago. Some folks want to export back into the folder they imported from. Others want a standard “Export” directory. On the principal that you can’t please all of the people all of the time, what you see is the compromise that was reached at the time. Also, IIRC, it was felt that complicating that preferences pane with something along the lines of “export to either the folder the audio was imported from or the folder where the project is but if the project hasn’t been saved then export to the standard export directory” was too complicated.