I use Windows7 64-bit installed in English. I installed from the .exe installer. I chose English as the installation language and in fact the installation dialog WAS in English. However, when I opened the program it appeared in French. I have tried reinstalling several times but problem repeated. I had no problem with two other installations I made on Linux and Vista 32-bit.
See here for how to change the language: http://manual.audacityteam.org/man/FAQ:About_Audacity#language