Veldora and Ifrit seems to have both been absorbed "alive" which is why they both seem to "live" inside Rimuru.

Everything else he's absorbed has been dead. Which I assume is why nothing except Ifrit has encountered Veldora.