A place being ritually pure and suitable to use for serving God is spoken of as if it were physically clean.
Here "they" refers to the priests, and "me" refers to Yahweh.