Or when at the end, you can overtype the seconds field in the Selection Start box, decrementing the required number of seconds.
Oh, and if you are doing fades you may want to try a newish plugin written by Steve Daulton called Pro-Fade (only for fade outs). I now use it all the time in preference tp the standard linear fade in Audacity, much better musical results. See this thread: https://forum.audacityteam.org/t/professional-sounding-fade-out/25297/1
And another tip: since you are doing a lot of these fades you might want to set up keyboard shortcuts for them, Audacity recently introduced user-settable shortcuts for effects, generators and analyzers (I use Alt + left-arrow for fade-in and Alt + right-arrow for Pro-fade-out). See this page in the manual for shortcut setting: http://manual.audacityteam.org/o/man/keyboard_preferences.html
WC