A new app that promises to pay people for recordings of their phone calls, which are then used to train AI models, has been disabled after a major security flaw was reported.
Neon is still in the top 10 of iOS free app downloads, but after TechCrunch reported Thursday about a security flaw that the news site found in the service, its servers have apparently been made unavailable to users.
The app can still be downloaded, but it's no longer functioning. It's unclear whether the service will return or how long it will take.
Emails to Neon Mobile, the company behind the app, have not been returned.
Don't miss any of our unbiased tech content and lab-based reviews. Add CNET as a preferred Google source.
According to TechCrunch, a flaw in the app allowed people to access calls from other users, transcripts and metadata about calls. The company notified Neon users that it was pausing the service but did not explicitly mention why, TechCrunch said.
Before the app was disabled, a legal expert warned about trouble it might cause, in addition to potential security flaws.
David Hoppe, the founder and managing partner of Gamma Law, which advises clients on thorny technological issues, told CNET that because some states have consent rules on recording phone calls, people using Neon should be very careful or avoid it entirely. Without certainty of its legality, he warned, "do not use this app."
Cash for calls
Neon is still available (at least for the time being) on iOS and Android. The company records users' outgoing phone calls and pays them up to $30 a day for regular calls or 30 cents a minute if the call is to another Neon user. Calls to non-Neon users pay 15 cents a minute. The app also offers $30 for referrals.
"You can cash out as soon as you earn your first ten cents," a Neon app FAQ says, "Once redeemed, payouts are typically processed within three business days, though timing may occasionally be shorter or longer."
Promo images for the Neon app on the iOS App Store promise money for phone call data but don't mention the data is used to train AI models. Apple App Store
The company promises it only draws from the recording of one side of the phone conversation, the caller's, which appears to be a way of skirting state laws that prohibit recording phone calls without permission.
While many states only require one person on a call to be aware that a call is being recorded, others, including California, Florida and Maryland, have laws that require all parties on a phone call to consent to recording. It's unclear how Neon functions with calls to those states. For Neon-to-Neon calls, two-party consent would presumably be implied.
The app doesn't record regular phone app calls, only those made within the Neon app or received from another person using Neon.
While the iOS version has shot up in popularity -- it reached as high as the No. 2 spot this week -- the Android version appears to be having some problems, at least according to some of the most recent reviews on the Google Play Store. The Android app only has a 2.4-star rating, and some user comments report network errors when people try to cash out on the Neon app.
Training AI using your data
According to the company's FAQ, the call data is anonymized and used to train AI voice assistants. "This helps train their systems to understand diverse, real-world speech," it says.
AI companies need increasing amounts of data to train their models, which may be why Neon is offering the monetary incentive.
"The industry is hungry for real conversations because they capture timing, filler words, interruptions and emotions that synthetic data misses, which improves quality of AI models," said Zahra Timsah, CEO of i-Gentic AI, which works in AI compliance.
"But that doesn't give apps a pass on privacy or consent," Timsah said.
Pushing legal limits
TechCrunch, which was one of the first sites to write about the app, pointed out that sharing voice data can be a security risk, even if a company promises to remove identifying information from the data.
Neon could be pushing its luck, especially across states and countries, when it comes to privacy and IP laws or regulations, depending on how it handles consent and where the data ends up.
"We don't know if there are sufficient safeguards to exclude the person on the other end of the conversation, but some level of consent would be required, and informing them of it being provided," said Valence Howden, a data governance expert and advisory fellow at Info-Tech Research Group.
Howden said that even if the data is anonymized, AI might not have a hard time retroactively discovering who is on the line in a Neon conversation.
"AI can infer a lot, correct or otherwise, to fill in gaps in what it receives, and may be able to provide direct links if names or personal information are part of the exchange," he said.
Can I be liable for call recordings?
Putting aside the requirements the Neon app had to meet in order to be included in Apple's App Store, it's reasonable to still have questions about the legality of recording phone calls, especially in states where all parties must consent.
That may be a major reason to avoid Neon, according to Hoppe, the legal expert.
"In the United States, it is not legal to simply record a phone call because an app's terms of service say you can," Hoppe said. "So, imagine a user in California records a call with a friend, also in California, without telling them. That user has just violated California's penal code. They could face criminal charges and, equally scary, be sued civilly by the person they recorded."
Violations, he said, could result in penalties of thousands of dollars per incident.
Hoppe said Neon's terms of service won't protect an app user if they face legal liability over recordings. And it doesn't help, legally speaking, that the person recording was paid for doing so.
"The user is the one pressing the record button," Hoppe said. "My strongest recommendation to anyone considering this would be: unless you are absolutely certain of the consent laws in your state and the state of the person you're calling, and you have explicitly informed and received consent from every other person on the call, do not use this app."