That is the default location of the Audacity folder for temporary data (the SessionData folder inside that folder stores the temp files).
That folder is not where Audacity settings are stored, which is C:\Users<user name >\AppData\Roaming\Audacity.
If C:\Users<user name>\AppData\Roaming\Audacity exists, which it will do if you ran Audacity before creating the Portable Settings folder, Audacity correctly does not delete that folder. But it should only be using the settings in Portable Settings.
With Portable Settings existing, change Snap To bottom left of Audacity to “Previous” and exit Audacity. Restart Audacity. Do you still see “Previous”?
Exit Audacity. Rename Portable Settings to something else, then restart Audacity. “Previous” should now have changed to “Off”. Exit Audacity, rename the folder back to Portable Settings and restart Audacity. “Off” should have reverted to “Previous”.
Gale