The installer should though ideally be able to set registry keys so that Audacity once installed is seen as a "Recommended Program" for particular file types in "Open with". Also the installer (or the Audacity interface) should ideally give the option to associate certain file types with Audacity.
The problem you have in this thread comes if you have had other instances of "audacity.exe" installed or located in the past in different locations, as you would if you had Audacity Beta then installed Audacity 2.0. As the Audacity Release Notes say:
It is a Windows limitation. The same issue arises if an older "open with" registry key still points to "audacity.exe", because the Audacity uninstaller doesn't remove open with keys (or give you an option to do so).the Audacity executable cannot be added to the Explorer "Open with" hover menu if you have another version of Audacity on the system which is also called "audacity.exe".
As far as I know this is not an issue if you install Audacity 2.0.0 over 1.2.6. 2.0.0 would by default install into the same %PROGRAM FILES% Audacity directory as 1.2.6 and the open with associations should continue as they were before.
In all these cases, the simple answer is to rename the 2.0 "audacity.exe" to some other name such as "audacity 2.exe" then you should be able to add it to the open with menu.
As an alternative to trying to sort problems like this out in the Windows Registry, which is dangerous, you could try
http://defaultprogramseditor.com/ or similar programs (search on Google for others). Or indeed in some cases a registry cleaner may help as rgdave confirms.
Gale