32 bit float to 16 bit PCM conversion process

Audio software developers forum.
Forum rules
If you require help using Audacity, please post on the forum board relevant to your operating system:
Windows
Mac OS X
GNU/Linux and Unix-like
Post Reply
jeffbiss
Posts: 4
Joined: Fri Mar 12, 2021 5:55 pm
Operating System: Windows 10

32 bit float to 16 bit PCM conversion process

Post by jeffbiss » Thu Sep 09, 2021 4:51 pm

I have a wave file open in a hex editor to get a good idea as to what's going on with the data, I have an all zeroes section that allows me to see how values are displayed in Audacity. The values in the displayed 32 bit float format are:

Code: Select all

01 00 02 00 03 00 04 00 05 00 06 00 07 00 08 00 
09 00 0A 00 0B 00 0C 00 0D 00 0E 00 0F 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
When I save that file as 16 bit PCM and open it to compare it with the 32 bit float I see that memory region as:

Code: Select all

01 00 01 00 05 00 01 00 08 00 04 00 07 00 09 00
09 00 09 00 0C 00 0C 00 0B 00 12 00 0A 00 04 00
FF FF FF FF 01 00 FF FF 01 00 00 00 00 00 FF FF
02 00 FE FF 01 00 00 00 00 00 00 00 00 00 00 00
Can someone point me to the Audacity file that converts 32 bit float to 16 bit PCM? I understand that the down conversion would result in value changes due to the lower bit count, but not what I'm seeing, especially when the values are zero. Also, I do not expect the down conversion to result in the "jagged" values, such as from 3 to 5 and 4 to 1. Audibly, these mean nothing, but I'd just like to understand what's going on in the code.

Thanks in advance.

steve
Site Admin
Posts: 82533
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: 32 bit float to 16 bit PCM conversion process

Post by steve » Thu Sep 09, 2021 5:59 pm

This is probably what's causing the confusion: https://manual.audacityteam.org/man/dither.html

If you turn off dither, then you'll probably see the values that you are expecting.
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)

DVDdoug
Forum Crew
Posts: 9609
Joined: Fri Sep 10, 2010 11:30 pm
Operating System: Windows 10

Re: 32 bit float to 16 bit PCM conversion process

Post by DVDdoug » Thu Sep 09, 2021 6:05 pm

I've never looked at the source code and I don't know how the bytes are arranged with floating-point WAVs...

But maybe this will help - The 0dB reference for regular (integer) WAV files is the maximum count (+32,767 or 32,768 with 16-bits). In floating point 0dB is represented by +/-1.0.
I do not expect the down conversion to result in the "jagged" values, such as from 3 to 5 and 4 to 1.
Are you talking about dither?

jeffbiss
Posts: 4
Joined: Fri Mar 12, 2021 5:55 pm
Operating System: Windows 10

Re: 32 bit float to 16 bit PCM conversion process

Post by jeffbiss » Thu Sep 09, 2021 7:09 pm

Thanks for your replies. I didn't know about dither, now I do. However, Audacity's conversion, Dither set to None for High-quality Conversion, didn't result in all zeroes where the original 32 bit float signal had zeroes:

Code: Select all

01 00 02 00 03 00 03 00 05 00 05 00 07 00 07 00
09 00 09 00 0A 00 0B 00 0B 00 0E 00 0C 00 11 00
06 00 FE FF 01 00 FF FF 01 00 FF FF 01 00 FE FF
02 00 FF FF 01 00 00 00 FF FF 03 00 FC FF 04 00
But now I know what to expect, a little fluctuation around higher order "zero" values at conversion.

steve
Site Admin
Posts: 82533
Joined: Sat Dec 01, 2007 11:43 am
Operating System: Linux *buntu

Re: 32 bit float to 16 bit PCM conversion process

Post by steve » Thu Sep 09, 2021 7:25 pm

jeffbiss wrote:
Thu Sep 09, 2021 7:09 pm
Dither set to None for High-quality Conversion, didn't result in all zeroes where the original 32 bit float signal had zeroes:
It does for me:

16-bit silence.png
16-bit silence
16-bit silence.png (67.82 KiB) Viewed 1577 times
32-bit-float.png
32-bit float silence
32-bit-float.png (68.45 KiB) Viewed 1577 times
9/10 questions are answered in the FREQUENTLY ASKED QUESTIONS (FAQ)

jeffbiss
Posts: 4
Joined: Fri Mar 12, 2021 5:55 pm
Operating System: Windows 10

Re: 32 bit float to 16 bit PCM conversion process

Post by jeffbiss » Fri Sep 10, 2021 2:59 pm

OK. I did this after I booted my computer from scratch and it appears that my issue is no longer happening. I tried the same process again and see no fluctuations, all values in the 16 bit PCM file in the "zero region" are the same as those in the 32 bit version. It must have been the state of Audacity that caused it.

Post Reply