Why did Joshua and Caleb say protection would be removed from the people of the land?

They said that the protection of the people of the land would be removed from those people because Yahweh was with the people of Israel.