I have no idea what the basis of the timeline in Audacity is.
A lot more about problems with audio on the Raspberry here:
https://hifiduino.wordpress.com/2014/11/13/raspberry-pi-b-digital-audio/
Problem number one: the internal 19 MHz CPU clock isn’t really suited for audio. That might not be your problem, but it’s part of the complexity.
The page I cited is talking about output, not input, but will give you some idea about the problem.
Problem number two: the USB implementation on the Pi.
Since they are talking about DAC’s and not about ADC’s, it isn’t exactly the same problem, but again, you’ll understand why everything also plays in a recording environment.
10% off seems like a lot. There are reasons why professional sound gear is so much more expensive. Clocks is just one of them. In a professional setting, you often need to sync several ADC’s. And that’s why a separate external audio clock is often used. To avoid the problem before it surfaces. Hec, there are even high end clock generators that don’t do anything but generate the master clock for the studio.
A very simple way to cut back on drift, is maker shorter recordings. I don’t know if that is an option?
Another solution could be to use a dedicated bat mic, like the Italian Dodotronic…
http://www.dodotronic.com/
And even here, they seem to have abandoned USB mics. They used to have this:
http://goedgeluid.be/Dodotronic-ultrasone-USB-mic.html