Sure he has potential, just not more than most. For rock lee to be able to be as good as other ninjas, he has to make his taijutsu so strong that it makes up for their ninjutsu and genjutsu. To use what you said, eventually, this will shorten his seconds on earth, because there's only so much you can subject your body to before it begins deteriorate and die. In my opinion, because of the toll chakra use and the gate opening thinggie takes on the body, the anime physics/biology does account for this. Now, if we're looking at potential as the amount of seconds he has to work with, Rock Lee would have to begin dipping in his reserve pool (old age seconds?) before other ninjas of similar current strenght would. Meaning, he'd die sooner, which gets back to him having less potential. This is of course assuming that everyone is right about rock lee, and not missing something about him that causes him to be far more than just another punk, something like an imortal body...