You can capture your microphone input and the system master output simultaneously onto separate tracks.

VirtualDJ utilizes hardware graphics rendering. Bandicam can target the VirtualDJ window directly via DirectX. This mode yields the highest possible framerate and isolates the application window. Select the in Bandicam.

Click on > Format Settings in Bandicam and apply the following tweaks:

Change the to your graphics card's hardware encoder (e.g., H264 (NVIDIA NVENC) or H264 (AMD APP) ) to ensure zero lag.

Use the Virtual DJ master volume and your audio interface to ensure your mix is loud but not clipping (avoid hitting the red in the Virtual DJ master meter).