GoKawiilSign up to see the future, today Can’t-miss innovations from the bleeding edge of science and tech Email address Sign Up Thank you!
If you ever feel like spilling your guts to ChatGPT, just know that you may have an audience.
404 Media reports that OpenAI is hiring hundreds of employees to sift through and read users’ prompts — including messages that contain sensitive personal information.
This isn’t part of a moderation effort. Instead, by reviewing the prompts, OpenAI hopes to improve the responses that ChatGPT gives. The contractors rate and critique each of the AI’s replies, with internal documents showing a focus on nudging ChatGPT away from anthropomorphizing itself and to be less sycophantic, 404 found.
Both qualities — anthropomorphizing and sycophancy — have been heavily scrutinized for their role in so-called AI psychosis, cases in which an AI incites and drives a user’s delusional behavior. Numerous lawsuits allege that an older, less restrained version of ChatGPT powered by GPT-4o drove their users to suicide.
Still, there are tradeoffs. Many users would probably find this surveillance of their chats, part of an operation dubbed “Project Lily,” to be an alarming invasion of privacy — if they had any idea this was happening.
Asked if they thought ChatGPT users knew about humans reading their chats, one of the employees replied: “No.”
“I don’t think they would imagine some contractor somewhere… is analyzing the conversations,” they told 404.
It’s unclear how many prompts are being read or how OpenAI decides to select them. But some of them include chats in which the user explicitly asks ChatGPT to keep their conversations private, 404 found.
The prompts that’re selected are anonymized, but sometimes include personal information. The dashboard that the employees use also includes a “user memories summary” that can give details about previous conversations, which can produce details on where the user lives. When 404 reached out to OpenAI, the company said it used a “Privacy Filter” model to anonymize conversations, which its website states “can make mistakes.”
... continue reading