How would he know? He may be a member of the Roger pirates, but he's not psychic.
Brook's captain and the rest of the sick crew set out over the calm belt and were never seen again. Unless people knew they'd fallen ill, most people would think they just ran from the Grand Line.