Do you use virtualdub to mux the image sequence and audio into an AVI container?
Use virtualDub's external encoder feature instead (the 1.10.3 beta). It's going to have the latest encoders. Handbrake is still using old versions of x264 and there has been encoding improvements in the later versions, especially for the new AVX2 instruction set.
And then there's the choice of AAC encoders. QAAC/NeroAAC/FAAC. Pick your poison.
For an mp4 muxer, mp4box is the solution.