So this whole time Don Flamingo was controlling his own puppet?

Plus Luffy got smacked hard. He let his guard down.