Those who carried burdens
These were people who carried supplies to those who were actually working on the wall.
each did his work with one hand, and with the other hand he held his weapon
This is probably an exaggeration to say that they always had their weapon with them.