You can't just eliminate the emotions involved from the equation. He was unwilling in all of this. His father sent him away because his father was defeated. His brothers died at the hands of Starks, and he clearly cares about his family. So yeah, he was unloyal to the Starks, but Rob should've taken these things into consideration. More than Theon being a traitor, Rob is an idiot.