Could be a poor connection on the iPod jack socket - or a damaged lead.
One of my iPods requires me to jiggle around the jack for the earbuds to get sound in both ears.
I have had my iPods connected successfully (with stereo and both channels working) to both my hi-fi setups in the manner you suggest - so it should work.