He's the son of the man who threw Victorica into the cell, so it's natural he's an asshole. But on the other hand, he seems to be granting Victorica more freedom than their old man would be willing to. Perhaps he's partly saying that to Victorica in order to coax her seek love. He has made no attempt to keep Kujou from her, after all, so he's not really that rotten.