The most current and familiar examples of adversarial AI are attacks on vision systems. These systems are becoming more pervasive—facial recognition unlocks computers or phones, opens doors, and is a key tool of the surveillance state. It’s no surprise there’s a lot of work going on to attack the models.
At the simplest level, academic researchers and activists hope to defeat facial recognition by either resisting recognition, or undermining the underlying models. Interestingly, both activists and academics have at least one objective in common: beating the system with minimal work.
The activist approach is straightforward: your face can’t be recognised if it can’t be seen–but wearing a Guy Fawkes mask to every rally, invites other kinds of attention, as would blinding cameras with bright LEDs (neither of which count as adversarial models since they defeat image capture, rather than analysis).
It’s much better to use something less noticeable. Late last year, a group of researchers published research in which tortoise shell spectacles fooled facial recognition systems’ AI models, causing the system misidentify people wearing the glasses.
Adding ‘noise’ to the image made the AI think a panda was a gibbon. Credit: Open AI
In the physical world, it meets both the academic and activist requirement for a minimalist attack. More importantly, for this discussion, it’s an example of a true adversarial attack—one that lets the system capture facial information, but tricks the model trying to analyse the image.
This post at Open AI offers a very good explanation of adversarial models attacking AI vision systems. “Starting with an image of a panda, the attacker adds a small perturbation that has been calculated to make the image be recognised as a gibbon with high confidence”.