Somehow I don't get the feeling from Kagura that she simply wants to put Jellal back into a prison, regardless of if she knows the council scumbags were going to execute him to hide the evidence of their own incompetence for good. So, if her intention is to slay him fair and square, she's fully prepared to face the consequences. However, I have a feeling she doesn't want to endanger her guild in the process, and that's the reason why she stopped Millianna and instead chose to bide her time, possibly until this whole contest business is over.
But of course I could be judging her wrong, especially if she expects Jellal to be quickly executed once caught.