Sure, the right idea. I have a feeling he wouldn't have been able to trust those arrogant and all-seeing Hyuuga too long either. The Hyuuga can't even trust each other, so how would they expect others to trust them? They would have been better off dead next. The Aburame clan is little better. They live in some closed-off village and they only care for their insects that can neautralize chakra. Highly suspicious and dangerous. Better off dead. The Sarutobi family likely dreams of getting back to power, so Konohamaru ought to be neutralized while it's still easy. The members of the Nara clan are far too smart, nobody can imagine what they are thinking behind the mask of laziless. They must be wiped out before they can execute their plans. The Inuzuka clan makes excellent ninja dog trainers, but dogs aren't supposed to think for themselves, only obey, so they could use some heavy pruning as well.
Like DE already said, this was never about saving Konoha. It was about saving the power structure of that time. Danzo was like any paranoid dictator has ever been, always looking for enemies of the state everywhere. When the leadership is like that, dissidents are sure to appear naturally.