First, Konohamaru, who we know is related to Sarutobi doesn't wear the symbol, so it's not a family thing.

Second, compare pictures of Sandaime when he was young with Asuma, they don't look alike at all. In fact Sandaime looks more like Shikamaru's dad.