"when they were forced to move to Babylon" or "when the Babylonians conquered the Israelites and made them go live in Babylon"
Here this means the country of Babylon, not just the city of Babylon.