Why didn't the wicked men think that the recovered booty should be shared with those men that were left behind?

They said those left behind did not go and fight with them.