GoKawiilMotivations:
Maybe you’re a Claude code/codex user diligently avoiding uploading personal data to LLM providers. Is it possible that the most valuable information isn’t your data- but the metadata about your sessions? The intuitions you apply in coming up with ways to coax the ai into solving problems might actually be special. It’s statistically improbable, but Claude might not be gaslighting you. It may be that you’ve actually got a real insight! Your agent sessions are transcripts of the hardest problems you work on. What would it cost you if someone had copies of them?
Last week there was public drama that shines a light on the risk that inference providers are training on user activity with the intent of delivering new discoveries. The mathematicians affected have published concerns about the ethics of frontier providers. If you missed it: https://www.theverge.com/ai-artificial-intelligence/991710/openai-navier-stokes-solution
When ‘EDR’ becomes Ethical Deflection and Refusal:
It’s become evident that the frontier providers are not only untrustworthy- but actively devious. If you want to protect your ideas, you cannot run inference on someone else’s hardware. It appears to be the case that everything you do with a frontier provider will get stolen. When a frontier provider talked through the navier-stokes equation situation with their lawyers, the best defensive strategy they came up with is “Cannot rule it out.” We can’t audit their retention or their training pipeline. Apparently neither can they.
These people should not be considered partners. They are pirates. If privacy matters, the only solution that enables verifiable protections is to operate your own hardware.
Hall Monitor as a Service
I am bewildered by OpenAI and Anthropic’s grandstanding on cybersecurity. They marvel at what they have wrought: AI beat their non-existent security controls. We are all in great ‘danger.’ Meanwhile, their llm “researchers” are running unsandboxed fleets of agents that appear to “spontaneously collaborate.” Somebody fetch me my fainting couch.
Frontier providers can afford advising from experienced security people. They almost certainly are paying some of them for perspective and leaving real cybersecurity guidance out of their public statements. All of this pearl clutching must be meant to solve a different problem than security.
The strongest & most accurate claim defenders can make about security is that we’ve found ways to make it “Pretty Hard” for attackers. This seems to result only when firms pay top dollar for the best talent in both exploit mitigation and exploit development. There’s rowdy but friendly competition between the defense and exploitation teams, and eventually you get controls that make successful attacks so expensive that they’re not worth doing. This is for the top tier companies in the country- although Microsoft seems like they’ve forgotten some lessons.
... continue reading