I have a RealTek stock computer sound card. I have taken a double ended male cable and done the latency test exactly as decribed in the manual at least 20 times. The latency keeps moving around 5-15 milliseconds both ways. The average is falling somewhere around -170. I have the buffer set to the default 100. In the example in the manual they showed it being off 0.1 milliseconds but mine is off 5-15 milliseconds each time. I subtract or add this amount each time and test again and it is always off. Is this due to my cheap stock sound card? 
Probably yes.
Set it to the average value.
Unfortunately some cheap sound cards can drift quite badly, though for short recordings it may not be noticeable when you listen back.