Noisegate: a differential-privacy gateway for untrusted AI agents
Give an AI agent query access to sensitive data, with a mathematical guarantee that no individual's record can leak — even if the agent is wrong, manipulated, or adversarial.
A recorded Claude Desktop session (replies trimmed; the chart cards are the session's own). An AI agent breaks 20 patients down by diagnosis, and the ±12 noise swamps every bin. Reminded that it cannot turn the noise off, it drains a three-answer budget until the gate returns a refusal instead of a quieter answer. On the 32,561-row census, a too-narrow slice is rejected at the trust boundary, while a full education breakdown comes back clean at scale. The refusal and the rejection are the live gateway's real enforcement, reproduced by python scripts/render_demo_gif.py . Try it from Use it from Claude Desktop.
At a glance
Working attacks, not claims. Three classic privacy attacks (differencing, membership inference, and singling-out by re-identification) run against the system's own engine. Each is shown succeeding with privacy off, defeated with privacy on, and regression-tested in CI so the defense cannot quietly rot.
Three classic privacy attacks (differencing, membership inference, and singling-out by re-identification) run against the system's own engine. Each is shown succeeding with privacy off, defeated with privacy on, and regression-tested in CI so the defense cannot quietly rot. Independently verified math. The hand-rolled noise mechanism matches OpenDP, the industry reference implementation, in all 35 noise-scale checks to within 1e-9, and 500,000-sample distribution tests pass with a positive control proving the test can fail.
The hand-rolled noise mechanism matches OpenDP, the industry reference implementation, in all 35 noise-scale checks to within 1e-9, and 500,000-sample distribution tests pass with a positive control proving the test can fail. 3× the utility from better accounting. Hybrid zCDP composition admits 308 queries where naive budget accounting stops at 100: the same privacy guarantee, three times the questions.
Hybrid zCDP composition admits 308 queries where naive budget accounting stops at 100: the same privacy guarantee, three times the questions. Built for AI agents. Runs as an MCP server for Claude Desktop. The connecting agent is untrusted by design, and every privacy property is enforced below it.
Runs as an MCP server for Claude Desktop. The connecting agent is untrusted by design, and every privacy property is enforced below it. A small, deterministic trust boundary. The LLM proposes queries; a boring, fully tested validation layer disposes. Nothing the model outputs can widen its own authority.
The LLM proposes queries; a boring, fully tested validation layer disposes. Nothing the model outputs can widen its own authority. Stack: Python · DuckDB · FastAPI · Streamlit · MCP SDK · Docker · GitHub Actions, with a 250+ test suite in CI.
... continue reading