He was quiet as they both went on, the bits of sadness and sympathy only serving to get under his skin, reminding him of all the reasons he hated being trapped in this place. He should have been able to find some comfort in the fact that his uncle wouldn't have left if he'd had the choice, but instead he couldn't get past the Castle tearing them apart in the first place.

And so, it is this thought that causes him to snap a little at Katara's question.

"The Castle knows exactly what it's doing! It works to make you as miserable as possible, without ever telling you what it wants from you!"

And by 'you' he of course means himself, since the Castle is obviously out to get him.