There is no point in recording higher than 96 kHz. Doing so will increase the file size but adds no additional audio information and data errors become more likely at higher sample rates.
You may find that there is a very subtle difference between recording 44100, 48000, or 96000 Hz, but higher does not necessarily mean better. You are most likely to find no audible difference at all.
A bit about the theory behind this.
The sample rate governs the highest frequency that can be represented. The absolute maximum frequency that can be represented is half of the sample rate (known as the “Nyquist frequency”). In practice, the highest frequency that can be represented is a little less than the Nyquist frequency, but with modern digital filtering technology they can get very close to the theoretical maximum. The highest frequency that human hearing is capable of is usually quoted as 20000 Hz, though this figure refers to “perfect” hearing for a person under 18. The upper limit declines with age, so a more realistic figure for an adult with good hearing is more likely to be 16000 Hz or less. Thus 44100 Hz sample rate is just enough for the full frequency range that young people with perfect hearing can hear, and is more than enough for the rest of us.
Analogue to digital converters generally have “native” sample rates that the hardware is designed to work at. This may be just one specific sample rate, or a few different sample rates. The most common “standard” sample rates for audio are 44100 Hz and 48000 Hz. Recording at a non-standard sample rate is achieved by converting the sample rate from that implemented in hardware to the required sample rate. Conversion is usually pretty good quality, but can be avoided altogether if everything is working at the same sample rate.
Audio CDs are always 44100 Hz, so if the recording is not 44100 Hz sample rate, then it has to be converted.
For best quality, the number of conversions should be kept to a minimum. Thus, if the hardware runs at 48000 Hz, then you can either record at 44100 Hz (one conversion) and stay at 44100 for the rest of the process, or record at 48000 Hz and convert once to 44100 Hz at some point before burning to CD. The sample rate conversion in Audacity 2.0.5 is very high quality, so if recording at 48000 it is generally best to export from Audacity as 44100 Hz (Audacity handling the conversion) rather than exporting at 48000 and using the CD burning software to do the conversion.
In theory there is no benefit to using sample rates greater than 44100 Hz. In practice, the best results are likely to be either, recording at 44100, or recording at 48000 or 96000 and changing the “Project Rate” (lower left corner of the main Audacity window) to 44100 before exporting. Most likely there will be no noticeable difference between these options other than file size, in which case recording at 44100 Hz is better (less demanding on computer resources).
Bit depth:
32 bit float is better for processing, so this should normally be the selected format for recording. The sound card will then supply the highest format that the system (as a whole) supports.
For CD, the exported file should be 16 bit as this is required for audio CDs.
For “perfect” quality back-up copies, the audio may be exported as 32 bit float WAV. Files in this format will be double the size of normal 16 bit WAV but will be very slightly “cleaner” in that it is an exact copy of the data in Audacity (not converted).
16 bit WAV (or Flac) is usually good enough for back-up copies, but 32 bit float gives a perfect copy. For this option you need to select “other uncompressed formats” as the file type. See these links:
http://manual.audacityteam.org/o/man/file_export_dialog.html
http://manual.audacityteam.org/o/man/wav_aiff_and_uncompressed_export_options.html
