When you say loud I assume you mean it is "clacking" more than usual etc and not exhaust noise. You might want to check your glow plugs to see if they are staying on long enough to properly help the engine to start. If you loose one or two the engine will still start, especially this time of year with the warmer weather, but will make more of the noise I think you are hearing. Someone that owns an 05 might be able to tell you how long the gp light stays on this time of year. I would guess around 4-5 sec?
It really won't warm up much on idle - do you have the increased idle feature engaged for the cooler temps? That will help a tiny bit. You are probably right it is just the temps - I lived in -40+ F temps - should have heard my '05 fire up then! Bet an oil change will help a bit, too. How many miles?