I've only seen the anime up to this episode and don't read the one piece manga, so this isn't a spoiler.
But I suspect the 7-9 geezers in Mariejoire(?) are the last survivors of the group who is behind the void century and the associated war. They somehow survived a few centuries* and the world nobles are their descendants and their authority derives from the old guys.
* They find someone with Jewelry Bonney's devil fruit power and make them rejuvenate them every few decades?
If this turns out true, I claim all the cookies.