Sample Data Export format

[steve]

@StevenABrown

do those decimal numbers represent the 24-bit PCM binary sample values exactly? Is there any rounding or approximation that occurs when the binary sample values are converted to the decimal numbers in the text file?

The values are rounded to 5 decimal places.

If my maths is correct, to represent normalized signed 24-bit numbers accurate to the least significant bit, you will need 9 decimal places.

For greater precision:

1. Find the “sample-data-export.ny” file.
2. Make a backup copy of the file somewhere convenient.
3. Open “sample-data-export.ny” in a plain text editor (such as NotePad++)
4. Find the line:

(setq *float-format* "%1.5f") ; 5 decimal places

5. For 9 decimal places, change that to:

(setq *float-format* "%1.9f") ; 9 decimal places

Note that for real-world recordings, anything beyond about 5 decimal places is likely to be just random noise.