Then you have the computer and Audacity set up correctly.
Now for the bad news. 
What you would normally do, is to have the computer and Audacity set up as above, then use the mixer to route both the main out and the 2 track input through to the headphones so that you have “direct monitoring” (zero latency) of what you are playing, combined with playback from the computer.
I’ve been looking through the Alesis Multi-mix manual and sadly it appears that Alesis have missed out that feature. There appears to only be a switch for monitoring either the main out “or” the 2 track in. This is a problem for overdub recording with Audacity.
Your options:
What Alesis expect you to do is to use a DAW that supports ASIO (such as Cubase, Reaper, Sonar…)
With ASIO you can set up routing of the input and output through to the USB in the computer. Essentially this is the same as using “Software Playthrough” in Audacity, but because ASIO is a lot more efficient than the standard Windows drivers, the amount of delay is much shorter, typically less than 20 ms rather than half a second for Audacity. Unfortunately Audacity cannot be shipped with ASIO support due to licensing restriction. So option 1, use a DAW that supports ASIO instead of Audacity.
Option 2. Connect the “Main Out” and the “Control Room Out” to any audio device that can accept 2 simultaneous inputs and has a headphone out. This could be an audio amp or a mini-mixer or similar. This is the hardware that (to keep the cost down) Alesis have apparently missed out.
Option 3. Don’t bother with monitoring the live input through your headphones. If you are using a microphone into the mixer, then presumably you can hear the sound source (your voice or whatever) acoustically. Some artists prefer to monitor in this way and listen to the previous tracks through one side of the headphones leaving the other ear uncovered.