On Windows, Audacity 2.1.0-alpha imports the file correctly using FFmpeg 2.2.2 but Audacity 2.0.5 using FFmpeg 0.6.2 adds padding as you describe. Your Audacity 2.0.4 will only be using FFmpeg 0.6.2 or earlier because that is all it supports.
So I suggest you update your obsolete version of Audacity by building and default-installing FFmpeg 2.2.x or 2.3.x then build the 2.0.6 source tarball of Audacity: http://audacityteam.org/download/source.
2.0.6 should then link against the new FFmpeg. You may need to configure Audacity thus:
./configure --disable-dynamic-loading
If you use FFmpeg 2.4.x or higher you may need to apply this patch or similar to the Audacity source to ensure that Audacity builds against FFmpeg:
--- src/FFmpeg.h 2014-08-27 01:33:29.601443000 +0200
+++ src/FFmpeg.h 2014-08-27 09:20:53.045146000 +0200
@@ -688,7 +688,7 @@
FFMPEG_FUNCTION_WITH_RETURN(
AVOutputFormat*,
av_oformat_next,
- (AVOutputFormat *f),
+ (const AVOutputFormat *f),
(f)
);
FFMPEG_FUNCTION_WITH_RETURN(
@@ -755,7 +755,7 @@
FFMPEG_FUNCTION_WITH_RETURN(
int,
av_fifo_size,
- (AVFifoBuffer *f),
+ (FF_CONST_AVUTIL53 AVFifoBuffer *f),
(f)
);
FFMPEG_FUNCTION_WITH_RETURN(
@@ -801,7 +801,7 @@
FFMPEG_FUNCTION_WITH_RETURN(
AVDictionaryEntry *,
av_dict_get,
- (AVDictionary *m, const char *key, const AVDictionaryEntry *prev, int flags),
+ (const AVDictionary *m, const char *key, FF_CONST_AVUTIL53 AVDictionaryEntry *prev, int flags),
(m, key, prev, flags)
);
FFMPEG_FUNCTION_WITH_RETURN(
Uninstall the 2.0.4 repository build of Audacity if you install 2.0.6 after building it.
Gale