LAME vbr mp3 export wasting space

[vinylivo]

Hello,

just by chance I realized that there is something wrong with VBR mp3 export with the lame library. this problem not only appears with the current version 3.82, but shows similar problem with 3.97. I checked both versions of 'libmp3lame.dylib'. I'm using Audacity 1.3.13.a, but checked with 1.3.12.beta without difference.
I'm running Snow Leopard 10.6.4.

recently I started to export mp3 directly from audacity with the 'export multiple' feature. before I always used aiff files and then iTunes-Lame, but for some reason the exported aiff files from audacity generate a nasty error-burst in iTunes at every track beginning and therefore are no option. it's also easier if track-marks (labels) are done within audacity anyway, so I don't have to type in track names twice. the remaining ID3 tags are easily managed by the metadata editor.

so when doing this mp3 export, I typically use 'preset-medium' for the files for my jukebox. this normally should generate between 140kbps up to 180kbps. but all the recent files were at least some 10% bigger (more than 160kbps and up). I checked with the iTunes-Lame version (3.97) I use together with iTunes, and all the files there were smaller. similar space problem when using 'preset-standard' but not as extreme. the other VBR selections (checked 5 and 3) are wasting space as well.
interestingly the mp3 export works correct when using fixed CBR export like 192kbps. no problem with these.

now the question was do we have higher quality?
I mean I use 'vbr preset-medium' to save space on my iRiver jukebox, because it only has a 40GB harddisk which is always full . otherwise I could use 128kbps CBR, but the quality is too lousy for me.
When searching for answers I got to a special mp3 tool called MP3packer, see http://omion.dyndns.org/. this crawls through the file and repacks the bit-reservoir for optimum file size. and see, when processing the vbr-mp3 files exported from audacity the filesize decreased by about 10%-12%

so why is the 'libmp3lame.dylib' doing that? or is it some comandline options passed from audacity?
when looking at the mp3-file in a text editor, you see a lot of 'LAME3.98.2' text paddings, looks like in almost every frame. perhaps it's encoded with the bit-reservoir off to make cutting easier, but why? after running the files thru the MP3packer, the 'LAME...' paddings previous seen are gone. and I'm sure the program doesn't steal any bit of the sound, it just packs the data correct.

so does somebody have some answers to this problem?

perhaps it's not even related to MacOS X? I can't check this now because I don't a have a Windows machine.

[kozikowski]

Quick note that "lame" is a UNIX standalone program. You don't have to use it with the Audacity wrapper. You can use it from the command line and there are many hundreds of options and settings.

man lame [enter]

http://linux.die.net/man/1/lame

Koz

[vinylivo]

Quick note that "lame" is a UNIX standalone program. You don't have to use it with the Audacity wrapper. You can use it from the command line and there are many hundreds of options and settings.

I know.
Actually the version used with audacity is a dynamic shared library (.dylib) on mac and a dynamic link library (.dll) on windoze.
And I use an original 'commandline' version via iTunes-LAME. I even tried direct commandline encoding.

The THING here is that I get about 10-12% bigger file sizes when I use the lame library with audacity compared to the other encodings, for the same input and the same lame versions (3.97). all encodings with "--preset-medium", nothing else. I never use any fancy options.
They should be (more or less) the same bitrate/filesize.
So this just doesn't make any sense at all...

[kozikowski]

If you're waiting for Audacity to always make sense, you may want to get some coffee.

Koz

[vinylivo]

If you're waiting for Audacity to always make sense, you may want to get some coffee.

unfortunately I don't like coffee at all. It smells fine but drinking, no. exception with a good dose of alcohol

[steve]

The THING here is that I get about 10-12% bigger file sizes when I use the lame library with audacity compared to the other encodings, for the same input and the same lame versions (3.97).

When you export from Audacity the source data is not identical to when you encode an exported wav file.
Audacity runs all exported data as a "mix", which is performed in 32-bit float format, but AFAIK Lame does not support 32-bit float PCM input, so Audacity is certainly doing "things" to the audio data on its way to LAME and I'd guess that the data sent to LAME from Audacity is not identical to the data in a previously exported WAV file.

The issue gets more intriguing if you test export from Audacity with the "external encoder" option.
(I'm unsure from your description if you've tried this).
I'm using Ubuntu 10.10, and exported an audio track three times.
1) Audacity MP3 preset: Medium, JS, Fast
2) Audacity MP3 VBR: Q 4,JS, Fast
3) External Encoder (the SAME installed LAME): -V 4

Results:
All three MP3 files are reported by MediaInfo as:
LAME3.98r
Encoding settings: -m m -V 4 -q 0 -lowpass 17.5 --vbr-new -b 32

Files 1 and 2 are both reported as:
Overall bit rate: 83.1 Kbps
File 3 is reported as:
Overall bit rate: 79.1 Kbps

As you say vinylivo, it does indeed appear that Audacity is adding padding or using the wrong minimum frame size.
Running MP3packer with the option "-b 32" on the Audacity encoded file will set the minimum bitrate for each frame to 32 kbps (which I think is what should happen with Quality: Medium / -V 4 anyway) reduces the MP3 file size so that it becomes very close to the size of the MP3 file encoded with "external" LAME.

I'll forward this issue to Audacity Feedback and see if there's any comment from any of the developers.

[Gale Andrews]

When you export from Audacity the source data is not identical to when you encode an exported wav file.

Yes, that is an additional factor, it seems. If I export (using the standard "MP3 files" choice) a WAV as "Standard Preset, fast, stereo", this consistently produces 1 kbps higher overall bit rate when the WAV was imported as 32-bit float as against imported as 16-bit.

However when exporting as CBR, the resolution the WAV was imported at does not affect the overall bit rate of the exported MP3.

I'm using Ubuntu 10.10, and exported an audio track three times.
1) Audacity MP3 preset: Medium, JS, Fast
2) Audacity MP3 VBR: Q 4,JS, Fast
3) External Encoder (the SAME installed LAME): -V 4

Results:
All three MP3 files are reported by MediaInfo as:
LAME3.98r
Encoding settings: -m m -V 4 -q 0 -lowpass 17.5 --vbr-new -b 32

Files 1 and 2 are both reported as:
Overall bit rate: 83.1 Kbps
File 3 is reported as:
Overall bit rate: 79.1 Kbps

Running MP3packer with the option "-b 32" on the Audacity encoded file will set the minimum bitrate for each frame to 32 kbps (which I think is what should happen with Quality: Medium / -V 4 anyway) reduces the MP3 file size so that it becomes very close to the size of the MP3 file encoded with "external" LAME.

For me, adding the -b 32 option doesn't reduce the file size any further at all compared to not adding this option, so I presume the -b 32 is correctly applied by Audacity. As vinylivo says, all the "LAME version" instances and the empty space that follows them (when exporting using "MP3 files") is what seems to be adding to the file size. The multiple version instances and the spaces are not in the VBR files exported at the Audacity command line.

As suggested by vinylivo's tests, when exporting CBR an equivalent command-line export produces the same file size as standard export.

exported aiff files from audacity generate a nasty error-burst in iTunes at every track beginning

This is a concern, too. Do you mean there is a glitch noise and then the file plays OK? Are you exporting with the AIFF (Apple) signed 16-bit PCM" option? What sample rate are you exporting at? Which version of iTunes?

I'll see if Leland can get across to look at the MP3 and AIFF issues.



Gale

[steve]

For me, adding the -b 32 option doesn't reduce the file size any further at all compared to not adding this option, so I presume the -b 32 is correctly applied by Audacity.

I agree with your observation, but I think the inference that you are drawing is incorrect.

If MP3packer is run without the -b 32 option, then MP3packer uses a minimum frame size of zero. MP3packer will shrink individual frames to the smallest possible size to fit the data in the frame.

Setting the -b 32 option tells MP3packer to use a minimum bit rate of 32 kbps. I used this option because that is the smallest frame size that should be used by LAME -V 4. Running MP3packer with -b 32 should not reduce the frame size, so there should be no reduction in file size. (It's possible that running MP3packer without the -b option could reduce the file size slightly, even if the MP3 has been encoded correctly with -V 4).

The "padding" (indicated by the "LAME version" instances, and surpless 00's) indicates that Audacity is using frame sizes that are bigger than they should be. Running MP3packer with -b 32 shrinks them back down to the correct size, with a noticeable reduction in file size. This does not happen if LAME is used as an external encoder.

With CBR, the frame size does not (and should not) vary. Audacity gets the frame size correct in this case. It's only with VBR (which is the recommended format) that the issue occurs.

[vinylivo]

Do you mean there is a glitch noise and then the file plays OK? Are you exporting with the AIFF (Apple) signed 16-bit PCM" option? What sample rate are you exporting at? Which version of iTunes?

actually it's not a iTunes problem itself, it plays the aiff files correct. also gapless from a continous recording exported with multiple tracks. but when I encode those 16-bit aiff files with the external iTunes-LAME encoder the nasty glitches are in the beginning of every final mp3 file. years ahead this wasn't a problem, but I cannot reproduce this anymore now. so I guess it's a problem of the iTunes-LAME script or the lame encoder can't handle some of the stranger ID3 aiff chunks correct.

[Gale Andrews]

Running MP3packer with -b 32 shrinks them back down to the correct size, with a noticeable reduction in file size.

As does running without -b 32 reduce the file size and remove padding. I understand -b sets a minimum, but I read you to say that -b 32 explicitly makes the file size smaller. I assumed the padding cleanup does that and would operate whatever the -b value?

actually it's not a iTunes problem itself, it plays the aiff files correct. also gapless from a continous recording exported with multiple tracks. but when I encode those 16-bit aiff files with the external iTunes-LAME encoder the nasty glitches are in the beginning of every final mp3 file. years ahead this wasn't a problem, but I cannot reproduce this anymore now. so I guess it's a problem of the iTunes-LAME script or the lame encoder can't handle some of the stranger ID3 aiff chunks correct.

OK, so:
1. If you export from Audacity as WAV and convert to AIFF in iTunes, then use iTunes-LAME, did/does that still have the problem?
2. If you used the Audacity AIFF and converted that to MP3 with the native iTunes encoder (not LAME), did/does that still have the problem?



Gale