They will ask
The word "They" refers to the people of Israel or the people of Judah who are returning from Babylon to Jerusalem.
join ourselves to Yahweh
This language is used to explain a spiritual connection to Yahweh.
that will not be forgotten
"that no one will forget"