What do I do if they ask me to shuffle when I'm doing a control?

What do I do when I'm controlling the spectator's card and they ask to shuffle? I can't let them, but if I say no, it'll look suspicious. I've thought about other techniques, but this trick doesn't allow for a key card, a force, or any other type of technique. Thanks a lot!