How to test for whether Audacity is receiving a signal.
Open up Audacity. There are a pair of level meters (meter tool bar) near the top of the Audacity window

Right click on the right hand meter (the recording level meter) and enable metering. When you get a signal, this meter will jump into life. (Note that if you go into preferences to change anything, the meter may switch off automatically)
Setting up your sound card.
Many sound cards on Windows use the "Windows Mixer" to set the record and playback options, however, SoundMax sound cards usually use a custom interface instead. Look in the icons near the clock (you may need to switch the "auto hide" off so that you can see them all). You are looking for the SoundMax control panel - it may look like a loudspeaker, or perhaps a red and white icon. You need to set the recording input to "Line in" (check that your tape player is connected to the "line in" socket). You should be able to set the recording level in the same place.