GoKawiilSign up to see the future, today Can’t-miss innovations from the bleeding edge of science and tech Email address Sign Up Thank you!
Researchers at the British AI security startup Mindgard found that a simple prompt spurred ChatGPT to drop its most basic safety guidelines, in another example of how the guardrails surrounding even the most popular AI models can easily be circumvented.
Specifically, according to reporting from the BBC, they coaxed OpenAI’s model to generate gruesome photorealistic scenes depicting gore and sexual content. Mindgard’s technique only involved slightly changing a widely-shared prompt that was originally intended to produce humorous images. It involves asking ChatGPT to restore an attached photo without actually uploading one, and then telling it to generate a new image.
“This is a perfectly innocent-looking instruction to an AI, but the consequence is it generates very, very bad imagery and content,” Mindgard founder Peter Garraghan, a computer science professor at Lancaster University, told the BBC.
Disturbingly, the prompts the researchers used didn’t specify the subject matter of the images. The AI, it seemed, produced the violent imagery “of its own volition,” Garraghan added.
Per the BBC, one picture showed a man with a large head injury. Another showed the corpse of a young woman in shorts and a crop top covered in blood, suggesting sexual violence. ChatGPT titled this image “grim crime scene aftermath.”
Another showed a frightened young woman tied up and gagged in an empty room, titled “abandoned in fear and restraint.”
While none of them showed real people, Mindgard has previously shown that ChatGPT could be tricked into creating nude deepfakes of specific persons without their consent.
Mindgard shared its findings with OpenAI, which only sent back an automated response. The company finally took action after Mindgard alerted the BBC, claiming it had addressed the issue.
“After investigating this trend, we’ve introduced additional safeguards against this type of prompt,” OpenAI told the BBC in a statement. It added that it has multiple layers of protection to stop users from making content that breaches its policies.
... continue reading