It wouldn't make much sense. Why would the village hate and distrust the child of their savior? And if he was the 4th's son, why don't the adults refer to him as the 4th's son? They always just call him the nine-tails kid.
Kishimoto originally planned for Naruto to actually be Kyuubi's son (it was in the pilot manga). So a leftover effect of that in the final version was that Naruto was shunned by the village.
Most likely Naruto was just some random orphan. Or, there is a remote chance that he is the 4th regressed to an infant state during the sealing, and only a few people know this (maybe Jiraya and the 3rd). This would explain why he has no family, but then again half the gennin kids in the village are orphans.