Rebounding isn't defense. By definition, defense is trying to stop the opposing team from scoring. Rebounding is trying to get the ball after you've stopped the other team from scoring. That's not defense, even if it occurs on the defensive end of the floor. Defense is what occurred before the rebound took place. Rebounding is a third category, separate from offense or defense.
Carlos Boozer was a bad defensive player, and a great rebounder. Those two things aren't mutually exclusive.
Right now Al Jeff is bad at both. Millsap is bad at defensive rebounding, good at offensive rebounding, and a much better defender than either Al Jefferson or Boozer.
This is why we need to start Elson, as he's the best rebounder on the team at the moment.
*facepalm