All 3 of them were alone.....but gaara was the only one who had absolutely nobody who loved him. People pretty much left naruto alone, and sasuke at least felt bonded with his family (and im sure the rest of the village after the incident)....but i dont know what could be worse than being told that nobody wants you to exist, even your parents!

edit: and to add to konoha rek's quote.....if sasuke is rubbed into it everytime he gets in a fight that he might lose, then gaara gets rubbed into it everytime he sees somebody care for somebody else (which im sure happens a LOT more frequently)