A more accessible way to set up latency correction

[Gale Andrews]

BUMP: can I just archive this thread - or is there an idea that can be expressed succinctly that I could transfer to Wiki>PFR with a link back to the archived thread. Advice please.

There are three "features" to vote on:

• Polishing and Wiki-publishing Robert's plug-in to measure time difference between clicks in two selected tracks (I think there are still some confusions in that plug-in that users could trip over, but it would be much better than not having the tool published).
• A built-in tool that can adjust the latency correction setting according to clicks being recorded from a track that is playing.
• "Automatic" correction that takes into account latency variability.
  • This could be an "improved" version of the disabled PortAudio correction that tries to detect the current round trip latency then corrects for it.
  • It could alternatively try to time-shift the recorded audio "intelligently" by looking for the first "audio" in the recording and aligning the point where that audio starts with the position of the cursor or selection start. So it would be looking for and pushing preceding recorded noise behind the align point. I think it if could be done it could be useful in some scenarios and could be an alternative "option" to the latency detection method.

I don't think any of us except Koz have ever voted for any of these, so vote away. I vote for an automatic method (including choice of method if practical).


Gale

[Robert J. H.]

BUMP: can I just archive this thread - or is there an idea that can be expressed succinctly that I could transfer to Wiki>PFR with a link back to the archived thread. Advice please.

There are three "features" to vote on:

• Polishing and Wiki-publishing Robert's plug-in to measure time difference between clicks in two selected tracks (I think there are still some confusions in that plug-in that users could trip over, but it would be much better than not having the tool published).
• A built-in tool that can adjust the latency correction setting according to clicks being recorded from a track that is playing.
• "Automatic" correction that takes into account latency variability.
  • This could be an "improved" version of the disabled PortAudio correction that tries to detect the current round trip latency then corrects for it.
  • It could alternatively try to time-shift the recorded audio "intelligently" by looking for the first "audio" in the recording and aligning the point where that audio starts with the position of the cursor or selection start. So it would be looking for and pushing preceding recorded noise behind the align point. I think it if could be done it could be useful in some scenarios and could be an alternative "option" to the latency detection method.

I don't think any of us except Koz have ever voted for any of these, so vote away. I vote for an automatic method (including choice of method if practical).


Gale

(how stupid, my lengthy comment has just been deleted, so a terze version of it)
I vote for the first two options.
By the way, the correction values can of course automatically be stored by my tool in preferences. It is possible that version 4 plug-ins can make use of the record command, thus, no additional tracks had to be selected, just one to be able to call the plug-in. This allows also multiple analysing and averaging.

The time shift or rather another alignment command, does not appeal to me, mostly because it is not flexible enough. Vocals often start after some seconds intro.
I can already do the described command with "Chain-it-up.ny". It removes silence from the track start. Therefore, alignment is easy if you've tapped along with a count-in measure and have deleted roughly to the beats in sync.

[Gale Andrews]

The time shift or rather another alignment command, does not appeal to me, mostly because it is not flexible enough. Vocals often start after some seconds intro.

Absolutely, it could only be an "option" for "automatic" correction, as I said. But it would work with the suggestions we've received for re-recording fluffs, whereby Audacity plays some audio before the fluff before it starts recording.

I can already do the described command with "Chain-it-up.ny". It removes silence from the track start. Therefore, alignment is easy if you've tapped along with a count-in measure and have deleted roughly to the beats in sync.

Yes but IMO, Chain-it-up is another of your super powerful tools that is really too complex to be generally releasable.

Would the "remove silence from track start" feature be useful as part of your latency measuring plug-in?


Gale

[Robert J. H.]

The time shift or rather another alignment command, does not appeal to me, mostly because it is not flexible enough. Vocals often start after some seconds intro.

Absolutely, it could only be an "option" for "automatic" correction, as I said. But it would work with the suggestions we've received for re-recording fluffs, whereby Audacity plays some audio before the fluff before it starts recording.

I can already do the described command with "Chain-it-up.ny". It removes silence from the track start. Therefore, alignment is easy if you've tapped along with a count-in measure and have deleted roughly to the beats in sync.

Yes but IMO, Chain-it-up is another of your super powerful tools that is really too complex to be generally releasable.

Would the "remove silence from track start" feature be useful as part of your latency measuring plug-in?


Gale

This can already be done by "Detach at Silences" and "Start to Zero", at least if it is totally white space. Otherwise, the user has to set a threshold value, as in my plug-in.

Two interesting things I've stumbled upon using 2.0.6 alpha.
Wasapi (loopback) seems still to have some problems. With both, Audio Buffer and Latency Correction set to zero, I get the following:
- A sample sound (1 s) gets 44100 + 441 samples long.
- There are 1395 zero-samples at the beginning
You can test it yourself:

Code: Select all

(dotimes (i 200000 )(if (not (zerop (snd-fetch s)))(return i)))

The user sees two main aspects of latency:
1. The latency while listening to the input in real time (play through)
2. Latency after recording--the track is shifted

The first point is pure prediction and is not our concern at this point.
The second one is a question of proper measurement as I do see it. Audacity should actually know how much time has elapsed from pressing "R" up to the time the recording is stopped. Or to be more specific, the times for the stream invoking and termination. Both, Portaudio and XWidgets have the appropriate counter functions.
The logical consequence is that the latency correction should be based on this timings and not on an arbitrary value in preferences.
An interesting site in this context is:
https://www.assembla.com/spaces/portaud ... Guidelines

Naively expressed: Audacity starts recording and then playback.
After the playback is stopped, we know exactly how many samples/frames were sent. The recording stream should naturally be longer (it began earlier). The difference in samples should therefore be the latency amount and can directly be deleted from the recording stream.
PA has some interesting get-functions for the latencies that count the time from the call back to the first/last sample at the DAC or ADC respectively.

There's a lot more in the above link. However, those are mainly guidelines for latency prediction and calculating a user-defined target latency.

[Gale Andrews]

The user sees two main aspects of latency:
1. The latency while listening to the input in real time (play through)
2. Latency after recording--the track is shifted

The first point is pure prediction and is not our concern at this point.
The second one is a question of proper measurement as I do see it. Audacity should actually know how much time has elapsed from pressing "R" up to the time the recording is stopped. Or to be more specific, the times for the stream invoking and termination. Both, Portaudio and XWidgets have the appropriate counter functions.
The logical consequence is that the latency correction should be based on this timings and not on an arbitrary value in preferences.

I agree - that's why my vote (as an ideal) is that this should be automatic detection.

An interesting site in this context is:
https://www.assembla.com/spaces/portaud ... Guidelines

Naively expressed: Audacity starts recording and then playback.

After the playback is stopped, we know exactly how many samples/frames were sent. The recording stream should naturally be longer (it began earlier). The difference in samples should therefore be the latency amount and can directly be deleted from the recording stream.

Thanks for the link.

I am not really sure what you mean. You of course don't mean - select the audio with overdub on, record, then measure how much longer the recording is, though that is part of it.

PA has some interesting get-functions for the latencies that count the time from the call back to the first/last sample at the DAC or ADC respectively.

Without really having the technical vocabulary to describe it, it would seem that PortAudio doing some "test" recording in the background "ought" to be a way forward.

You may want to read: http://audacity.238276.n2.nabble.com/re ... l#a2172312 from which you'll notice that inputBufferAdcTime wasn't set by PortAudio for most API's and mostly returned zero even if set. Input latency and output latency often returned zero except on ALSA. So the returned values couldn't be trusted and at that point we turned off the automatic PortAudio correction.

I presume nothing has changed in PortAudio but I have not followed it. The simple code changes contained in http://code.google.com/p/audacity/source/detail?r=8609 disabled automatic latency correction so you could turn it on and see how correction works now.


Gale

[Gale Andrews]

Two interesting things I've stumbled upon using 2.0.6 alpha.
Wasapi (loopback) seems still to have some problems. With both, Audio Buffer and Latency Correction set to zero, I get the following:
- A sample sound (1 s) gets 44100 + 441 samples long.
- There are 1395 zero-samples at the beginning

Similar in 2.0.6-alpha and 2.0.5 Release for me, and similar in other API's except that the start is noise rather than absolute silence.


Gale

[Robert J. H.]

The user sees two main aspects of latency:
1. The latency while listening to the input in real time (play through)
2. Latency after recording--the track is shifted

The first point is pure prediction and is not our concern at this point.
The second one is a question of proper measurement as I do see it. Audacity should actually know how much time has elapsed from pressing "R" up to the time the recording is stopped. Or to be more specific, the times for the stream invoking and termination. Both, Portaudio and XWidgets have the appropriate counter functions.
The logical consequence is that the latency correction should be based on this timings and not on an arbitrary value in preferences.

I agree - that's why my vote (as an ideal) is that this should be automatic detection.

An interesting site in this context is:
https://www.assembla.com/spaces/portaud ... Guidelines

Naively expressed: Audacity starts recording and then playback.

After the playback is stopped, we know exactly how many samples/frames were sent. The recording stream should naturally be longer (it began earlier). The difference in samples should therefore be the latency amount and can directly be deleted from the recording stream.

Thanks for the link.

I am not really sure what you mean. You of course don't mean - select the audio with overdub on, record, then measure how much longer the recording is, though that is part of it.

Thanks Gale.
I presume that the latency issue is only a problem for overdubbing. I've not yet studied the actual playback/recording code within Audacity, so my assumptions may be somewhat coarse.
While overdubbing, the tracks are played and at the same time another one recorded.
Those streams have to be invoked somehow.
As you've stated (it's in the link you've supplied too), some get-latency functions seem to return ironious values, (perhaps mostly for full duplex?).
There's also a PA function "get-current-time" that should return the current stream time and is suggested to be used for synchronization with e.g. Midi. If playback and recording are different streams, those times could be compared, against the system timer too.
The question is now, which values can be gathered to eventually calculate the exact latency in the sense of real elapsed time difference between playback and recording.
For example:
- the system timer can be read out before recording/playback and after playback/recording (or/and just before closing the stream).
- The total playback time should actually be known (we have the played buffers/frames/samples).

As a developer, I would start with gathering all possible values and writing them e.g. to the log file.
I am sure that some feasible correlation to our sought correction amount can be found.
Edgar would be the perfect candidate for this task, if no developer is interested anymore. Although I suspect that they've not looked at the latency issue from the physical point of view but from a "weather forecast" kind of one.

Code: Select all

A possible scheme
Legend:
R = recording stream
P = Playback stream
° = measuring point, if feasible
| = start or stop
- = buffer activity
. = latency

(user clicks Record with overdub enabled --> first °)
R ..°|---.--.---°.---.--.--.-°|° (stopped by user)
P ....°|--------°|° (out of play data, record goes on)

The points inside the recording stream should indicate that it is probably not as linear as the playback stream. That's always hard to tell with all that buffering stuff. It might be necessary to also read out additional values, such as the number of frames made ready for output. The scheme bases on the assumption that the record starts earlier than the playback, but this may not be true.

[Robert J. H.]

Two interesting things I've stumbled upon using 2.0.6 alpha.
Wasapi (loopback) seems still to have some problems. With both, Audio Buffer and Latency Correction set to zero, I get the following:
- A sample sound (1 s) gets 44100 + 441 samples long.
- There are 1395 zero-samples at the beginning

Similar in 2.0.6-alpha and 2.0.5 Release for me, and similar in other API's except that the start is noise rather than absolute silence.


Gale

I've only noise for real sources but silence for speaker loopback.

[Gale Andrews]

Two interesting things I've stumbled upon using 2.0.6 alpha.
Wasapi (loopback) seems still to have some problems. With both, Audio Buffer and Latency Correction set to zero, I get the following:
- A sample sound (1 s) gets 44100 + 441 samples long.
- There are 1395 zero-samples at the beginning

Similar in 2.0.6-alpha and 2.0.5 Release for me, and similar in other API's except that the start is noise rather than absolute silence.

Gale

I've only noise for real sources but silence for speaker loopback.

Do you get noise at the start for WASAPI with real source like stereo mix or line-in?


Gale

[Robert J. H.]

Do you get noise at the start for WASAPI with real source like stereo mix or line-in?

Hard to tell, my sound card has a terrible DC-offset of about -0.2. I can't judge if there's additional noise, apart from the faulty USB voltage.
It seems that the actual Audio starts after 3697 samples, as opposed to 1395 from the loop back.
That's of course the latency. However, It is rather strange that the recorded material is longer than the generated track but less than the Audio + latency. That's all neither fish nor bird.