The family certainly has very diverse members. But at least some of them realise a whipped dog won't make the best leader.
And the ones that don't probably don't intend for him to BECOME leader. I don't imagine his older siblings approve of the idea of him taking over some day.