Fluctuating latency

I have a fluctuating latency issues that I can’t figure out how to correct.

You mean when you are trying to set up latency compensation (http://manual.audacityteam.org/man/latency_test.html) the amount of latency keeps changing? How much is it changing by?
You are aware that if you use a different audio device, or make other changes to the set-up (such as changing the sample rate or sample format) that the latency will change?

Yes, the latency compensation process doesn’t snap it to the right time, just random times.

You mean when you are trying to set up latency compensation the amount of latency keeps changing? How much is it changing by?

Yes, it varies quite a bit each time I perform the latency compensation process.

How much is “quite a bit”? If it’s more than a few milliseconds, then you probably need better hardware.