Discrepancy in zoomed display of the region and of cuts

[Gale Andrews]

Some audio software places the "cut-off point" or "time boundary" effectively on the sample point, not between points. I still hold to the heresy this is less confusing than Audacity's "Sphere of influence" behaviour.

I think that Audacity putting clip boundaries on sample points (and snapping the cursor to sample points when "Snap To" snaps to samples) is confusing in the context of otherwise having the time boundary between samples.

Even more heretical, and possibly not thought through, why not snap to samples as default if you are not snapped to any other format, and avoid the confusing and ugly display problems noted here?


Gale

[waxcylinder]

Even more heretical, and possibly not thought through, why not snap to samples as default if you are not snapped to any other format, and avoid the confusing and ugly display problems noted here?

Ooh that's a cute workaround

[steve]

why not snap to samples as default if you are not snapped to any other format

In audio tracks we already do that, and I think that is the only sensible thing to do.

What we don't currently do when "Snap To" is not enabled, is to snap the selection (hence labels) to sample periods. Quantizing the selection to "sample time" is more problematic because there may be more than one sample rate at play. Consider if we were selecting in two tracks. One has a sample rate of 44100 Hz, one has a sample rate of 48000 Hz and the Project rate is 100000 Hz. Which sample rate "should" we snap to, or should we snap to 1/44100th second in one track, 1/48000th second in the other track, and 1/100000th second in the Timeline and label tracks? Currently, when "Snap To" is enabled we snap to the Project Rate.

The problem that we have when considering very short time periods, is that there is a discrepancy between "analogue time" and "digital time".
Consider a digital watch that displays seconds - there is nothing to distinguish the difference between the start of a second, the middle of a second and the end of a second. Unless we bias measurements and calculations based on our own (analogue) sense of time, the maximum precision of calculations will be 1 second.

In digital audio we usually represent samples as "dots", which in one sense is misleading because a digital "sample" has exactly the same value in both "magnitude" and "sample number" from start to end. In fact it makes little sense to think of "start and end" of a sample because "1 sample period" is the smallest quantum of time, so the properties of a sample are not "start time, start value, end time, end value", but simply "sample value and sample number". The value and number define what a sample "is".

When considering what the samples "represent", the picture is slightly different. Assuming "perfect" conversion between digital and analogue, the sample values represent points on a curve and the curve will pass through each sample as plotted on a graph as amplitude / time. This is how samples are represented in audio tracks in Audacity (and many other audio editors).

Some audio software places the "cut-off point" or "time boundary" effectively on the sample point, not between points.

It is arguable whether the "dots" should be at the start of the sample period, or the middle of the sample period (currently we put them at the start), and it is arguable whether to snap the selection to the dots, or mid way between the dots (we snap between dots). The important thing is that it must be entirely consistent so as to avoid "off by one" errors. We can see the drawback of selecting "between dots" in examples posted previously in this topic. Selecting "on the dot" has it's own drawback in that it is ambiguous whether a sample that is "on the line" is "in" the selection or "outside" the selection (which is not ambiguous if we select "between" dots).

Blind users will probably wonder what all this fuss is about, because they are not thrown off by the visual appearance. If we disregard the graphical representation, what we are left with is a list of numbers representing a sequence of sample values. When we "make a selection", we are selecting a sub-sequence from that list. An item from the list is either within the bounds of our selection, or not within the selection and there is no grey area.

[Paul L]

You guys are debating display and selection behavior now, not suggesting anything with implications for how I generate labels in plug ins... right?

I'm glad I'm not alone in finding the zoomed display confusing.

[steve]

You guys are debating display and selection behavior now, not suggesting anything with implications for how I generate labels in plug ins... right?

We've possibly drifted a little way off your original point, but it is still definitely related very closely.

Thinking of "samples" as an indexed list (actually an array) of values (rather than of dots on a graph) may help to avoid much of the confusion when writing scripts.
The "nth sample" is the "nth item" in the list (since elements of lists and arrays are numbered from zero, this with be the "n-1" element)
The first sample in the list is at time = zero, the second at time = 1/sample_rate, and so on.
To convert from "analogue" time to sample number, round to the nearest.

So, for example, to "read" the nth sample (the n-1 element in the array), the sample time is (n-1)/sample_rate (array elements are numbered from zero)

Code: Select all

;; generate 11 samples from 0 to 1
(setf s-array #(0 1 2 3 4 5 6 7 8 9 10))
(mult 0.1 (snd-from-array 0 *sound-srate* s-array))

Code: Select all

;; read the sample values
(abs-env
  (dotimes (i 11)
    (print (sref s (/ i *sound-srate*)))))

Or with a prettier output:

Code: Select all

;; read the sample values
(setq output "")
(abs-env
  (dotimes (i 11)
    (setq output
      (format nil
        "~asample # ~at element # ~at value ~a~%"
        output
        (1+ i)
        i
        (sref s (/ i *sound-srate*))))))
output

or if we want to "include" the 3rd, 4th and 5th samples with a labelled region, then we need to select from 2/sample-rate to 5/sample_rate

Code: Select all

("sr" = sample rate)

Sample#  Array#   time
1         0       0
2         1       1/sr
---- from here ----
3*        2       2/sr
4*        3       3/sr
5*        4       4/sr
--- to here -----
6         5       5/sr

So it doesn't much matter whether we select from 1.5/sr to 4.5/sr, or from 2.4999/sr to 5.4999/sr, as long as the quantized (rounded) values select from: 2/sr to: 5/sr

Code: Select all

(let ((sr *sound-srate*))
  (list 
    (list (/ 1.5 sr) (/ 4.5 sr) "")
    (list (/ 2 sr) (/ 5 sr) "")
    (list (/ 2.4999 sr) (/ 5.4999 sr) "")))

[Gale Andrews]

why not snap to samples as default if you are not snapped to any other format

In audio tracks we already do that, and I think that is the only sensible thing to do.

What we don't currently do when "Snap To" is not enabled, is to snap the selection (hence labels) to sample periods.

Yes I was suggesting to snap selections to sample periods even if Snap To is off.

But visually it's unconvincing to say that we snap selection to sample periods if Snap To samples is on. It looks like we snap to samples. If Snap To samples is off, you can drag the selection as finely as you can move it. If Snap To samples is on, you can only drag the selection so that it aligns with a sample point.This produces the ugly "gatelegged" appearance and the confusion Paul has described.

As another confusion, whether Snap To is on or off, you don't get the pointing hand when hovering over the selection edge in the waveform - the selection edge has effectively moved into the Timeline.

It's just as well the Manual doesn't try to document this.

Quantizing the selection to "sample time" is more problematic because there may be more than one sample rate at play.

Consider if we were selecting in two tracks. One has a sample rate of 44100 Hz, one has a sample rate of 48000 Hz and the Project rate is 100000 Hz. Which sample rate "should" we snap to, or should we snap to 1/44100th second in one track, 1/48000th second in the other track, and 1/100000th second in the Timeline and label tracks? Currently, when "Snap To" is enabled we snap to the Project Rate.

Consider this image:

gatelegged audio and label selections.png (18.46 KiB) Viewed 1403 times

I see that the selection in the 9000 Hz track is snapping between sample dots (fair enough on Audacity 's own terms) and because the audio track above is at a higher rate and also obeys snapping between sample dots, that selection is smaller.

In that image, I don't see why having to choose which sample rate the selection snaps to would prevent the selection in the label track also snapping in-between sample dots (if that is the choice we make).

Also I've known several people claim strongly that a scenario as above where selections are of different lengths in different tracks causes clicks when you cut, though I have never proved it. Looking at that image makes me wonder if the selection should be quantised to the lowest track rate so as to prevent potentially large time selection discrepancies.

Selecting "on the dot" has its own drawback in that it is ambiguous whether a sample that is "on the line" is "in" the selection or "outside" the selection (which is not ambiguous if we select "between" dots).

I don't think it's any more confusing than the question of where a "full" beat ends that we two amused ourselves with recently Certainly no more confusing that clicking to Snap To a sample aligns the cursor with the right edge of the dot.

If all selections were to snap to dots instead of between them, then I suggest colouring the samples that are included in the selection differently.


Gale

[steve]

I think that at this stage of the game, the chances of changing the relationship between selection time and samples is zero. Conversion between time and samples goes deep into the code As mentioned previously it must be consistent throughout so as to avoid "off by one" errors, and that has taken a huge amount of time and effort.

The one thing that could be done, is to just make a change in how we represent samples graphically. Instead of putting the dots at sample times, we could put dots in the middle of sample periods (which is the "cross-over" point that the selection snaps to). As an example, if we have a sample rate of 1 Hz: Currently the dots are shown at 0.0, 2.0, 3.0, 4.0 ... and selection "snap to" positions are at 0.0, 0.5, 1.5, 2.5, 3.5.... If the dots were moved to the middle of the sample periods, then the first sample would be at 0.5 rather than at 0.0. The dots would be at 0.5, 1.5, 2.5, 3.5... (the same as the "snap to" positions). This would require no code changes other than the code that displays the waveform, but I still expect there will be no developer interest (and quite probably there will be some opposition). There is still the conceptual problem of "does cutting a selection from 0 to 0.5 delete the first sample" (which is more or less where this topic started).

[Gale Andrews]

I think that at this stage of the game, the chances of changing the relationship between selection time and samples is zero. Conversion between time and samples goes deep into the code As mentioned previously it must be consistent throughout so as to avoid "off by one" errors, and that has taken a huge amount of time and effort.

I'm sure this view is correct, but may not necessarily be correct if there is a complete rewrite for any future mobile version of Audacity. A lot would depend how efficient the current scheme is at waveform display (the current slowness of waveform display is a major obstacle to a mobile version).

The one thing that could be done, is to just make a change in how we represent samples graphically. Instead of putting the dots at sample times, we could put dots in the middle of sample periods (which is the "cross-over" point that the selection snaps to).

The terminology is getting confusing here. Selection in the audio track snaps to this cross-over point (whether Snap To is off or on), but (if Snap To samples is on) selection in the label track snaps to the sample points.

As an example, if we have a sample rate of 1 Hz: Currently the dots are shown at 0.0, 2.0, 3.0, 4.0 ... and selection "snap to" positions are at 0.0, 0.5, 1.5, 2.5, 3.5.... If the dots were moved to the middle of the sample periods, then the first sample would be at 0.5 rather than at 0.0. The dots would be at 0.5, 1.5, 2.5, 3.5... (the same as the "snap to" positions).

The audio selection snap to positions, that are snapped to whether Snap To is on or off, yes?

What would happen to Snap To clicks which currently snap to the sample times (0.0, 1.0, 2.0 in this example)? Would they snap to the sample times as now, hence in-between dots if we moved the dots to the middle of the sample period?

This would require no code changes other than the code that displays the waveform, but I still expect there will be no developer interest (and quite probably there will be some opposition).

I would probably wager a few pennies on that too.

There is still the conceptual problem of "does cutting a selection from 0 to 0.5 delete the first sample" (which is more or less where this topic started).

And given that, and we would still have the "gatelegged" audio and label track problem, I am not sure if this helps overall or not.

Although I suggested that the selection in the label track should snap to sample periods, so removing the "gateleg", I would not make a complete fresh start from there. I would say that audio (and label track) selections (and clicks) snap to the sample times if not snapped otherwise. Dots are at the sample times.

Wavosaur has dots at the sample times and clicks and selections snap to the sample times. Goldwave does not seem to have sample dots but seems to snap clicks and selections to sample times.

CoolEdit is almost like Audacity. Dots are at the sample times but selections snap to the cross-over points between sample times. Clicks snap to the crossover points, which seems to make more sense in that context than snapping to the sample times. A half-sample period is shown behind zero which makes it clearer what is going on. Cue Ranges snap to the cross-over points. I think even this is clearer than Audacity.


Gale

[steve]

Selection in the audio track snaps to this cross-over point (whether Snap To is off or on), but (if Snap To samples is on) selection in the label track snaps to the sample points.

Yes, that is the current behaviour.

There are no restrictions about where the Timeline selection snaps to - this snap position can be at any interval that we care to define.
The snapping position in audio tracks must be between sample times because a sample must be inside or outside of a selection and there is no room for ambiguity.

If the dots were drawn in the middle of the sample period and "Snap To" is on, then if the selection in the Timeline/Label track snapped to the dots it would be snapping to the same position as the selection in the Audio track (assuming that the audio track has the same sample rate as the Project Rate).

Current behaviour with "Snap To" enabled:

• Dots mark sample times.
• Selection in audio track snaps to the middle of the sample period.
• Selection in the Timeline/Label tracks snap to the dots.

This combination produces a dogleg in the selection when the track sample rate is the same as the Project Rate - the audio track selection is offset from the Timeline selection by half a sample period.


Possible behaviour if dots marked the middle of the sample period:

• Dots mark the middle of the sample period.
• Selection in audio track snaps to the middle of the sample period.
• Selection in the Timeline/Label tracks snap to the dots.

The only thing that we have really changed is "what the dots signify". They now signify "sample periods" rather than "the time of a sample".
The selection in the audio track will now snap to the same position as the dots.
The selection in the Timeline still snaps to the same position as the dots.
When the audio track has the same sample rate as the Project Rate, there is no dogleg - the selection in the Timeline is in line with the selection in the audio track.

I'm not convinced that this would be better than what we have now, but it would "look" neater, provided that the audio track sample rate is the same as the Project Rate (which it usually is).

[Gale Andrews]

Possible behaviour if dots marked the middle of the sample period:

• Dots mark the middle of the sample period.
• Selection in audio track snaps to the middle of the sample period.
• Selection in the Timeline/Label tracks snap to the dots.

The only thing that we have really changed is "what the dots signify". They now signify "sample periods" rather than "the time of a sample".
The selection in the audio track will now snap to the same position as the dots.
The selection in the Timeline still snaps to the same position as the dots.
When the audio track has the same sample rate as the Project Rate, there is no dogleg - the selection in the Timeline is in line with the selection in the audio track.

OK so you now think that by moving the sample dots to the middle of the sample period, you could do what I asked for (have selections in the Timeline/Label tracks snap to the sample period crossover points, avoiding the dogleg) and without risking off-by-ones?

And do I assume by the same logic that mouse clicks in the audio track would "snap to the dots" as now, so would now snap in-between sample periods rather than at sample times?

if so, then we would have a scheme that is more consistent, intuitive and hence more documentable. So I would see it as a potential advance. It is not the Wavosaur model which I think is the easiest to grasp (providing you could visually demonstrate when a sample was in the Timeline selection and when not).

I'm not convinced that this would be better than what we have now, but it would "look" neater, provided that the audio track sample rate is the same as the Project Rate (which it usually is).

Subject to my questions above, it seems to be a considerable improvement. Clicks and both types of selections would snap to the middle of the sample period, and that position is where the sample dots are.

The big drawback is moving the sample dots away from the sample times, which is about the only thing that is easily grasped with the current scheme . There would probably be a mistaken assumption that we were snapping to sample times.

It could possibly be ameliorated by using a different representation than a dot e.g. ][ (essentially an I-Beam with some kind of impression that it was two boundaries attached to each other). If you imagine those in a waveform

Code: Select all

 ][   ][    ][                             ][  ][ 
                  ][    ][          ][    
                               ][  

it's not totally unconvincing (to me). But before voting for that (as opposed to "mouse clicks and timeline/audio selections snap to sample times, dots at sample times" which needs a complete makeover) I'd want to be sure there was no risk.


Gale