A few questions - which may be obvious to you but not for someone who does not know your equipment:
The “Pioneer DJM-S11” is a mixer, right?
What other equipment is involved (turntable, instruments, microphones, etc.)?
How is that equipment connected together?
What is “Serato Virtual Audio”? Why is it needed?
Any other things I forgot to ask?
Usually, any signal coming from outside can be recorded by Audacity (as long as Audacity has access to “Microphone” in the system settings. A MacBook has no proper analog sound-in. If your mixer does not provide a digital USB sound-out, you need an external USB sound card.