# My lips will shout for joy

"I will shout for joy" 

# even my soul, which you have redeemed

"and my soul, which you have redeemed, will sing praises" 
# my soul

Here "soul" refers to the whole person.