I think it was bc when he lived with her, they had nothing, but he was able to get away and his life changed for the better. He was mad that nobody cared enough to help her like people helped him.
But I may be wrong, that's just my understanding of that scene