Like Mut@t@ said, at the point of the story in the Forrest of Death where his eyes turned red the viewer/reader didn't know that he was stronger than average. The red eyes were just a suggestion that there was more to him than meets the eye, and that he had some sort of strength we didn't know about. From there, it was revealed about the spy factor, Orochimaru, etc.

If he really did have some super power, wouldn't we have seen it in a fight involving all 3 Sannin?