The loss/gain in muzzle velocity will vary greatly depending on your ammo, handloads, etc., but generally speaking you're looking at a margin of 35-40 fps variation for every inch of barrel gained or lost. Again these are common averages, but the extra 4" in barrel length will net you around a 140-160 fps gain in muzzle velocity. The extra length and resulting increase in muzzle velocity will gain you marked improvement in consistency at distances approaching 500-600m.
Another factor to consider is barrel resonance. The shorter 20" barrel is stiffer and will be less affected by vibration as the bullet travels down the barrel. In my own experience (take it for what it's worth), I print a lot tighter on paper with my 20" guns at 300m and in than I do with my longer barrels. I only break out the 24" when I plan on reaching out to 400 and out.
If you're shooting for pinpoint accuracy, do some reading on barrel thermodynamics. The heating and cooling of your barrel while firing is one of the most significant factors contributing to degredation in accuracy from one shot to the next...