GoKawiilDetection-as-Code Foundry pairs LLM agents with a deterministic referee to validate Sigma rules
A new multi-agent system called Detection-as-Code Foundry uses four role-specialized LLM agents—Threat Analyst, Blue Detection Engineer, Telemetry, and Red-Team Adversary—built on LangGraph, Pydantic, FastMCP and Claude to turn a threat input like a CVE or ATT&CK technique into a Sigma detection rule. The rule is run through an adversarial Blue-vs-Red loop and shipped with a proof bundle: a positive and benign test corpus, an adversarial mutation set, and a coverage label (real-coverage, brittle, or broken) with FP/FN counts and a reproducibility hash. A deterministic engine, not the LLMs, decides all pass/fail verdicts, evasion results and the deployment gate.
GoKawiil's interpretation of the reporting above, not reported fact.
By separating creative rule-writing (done by LLMs) from truth-determination (done by a fixed, repeatable evaluator), the system aims to make AI-generated detections auditable and reproducible rather than opaque guesses, which could address a longstanding trust gap in using generative AI for security engineering. The project's own framing suggests this is meant as a verified starting baseline for human detection teams to extend, not a claim of permanent, evasion-proof coverage.
- The system outputs three linked artifacts: a Sigma rule, a threat report with convergence history, and a proof bundle with coverage metrics.
- All truth-determining functions—FP/FN scoring, adversarial evasion testing, and the coverage verdict—run on a deterministic engine rather than the LLM agents themselves.
- Red-team testing uses a fixed set of two real attacks reapplied each round, intended to prevent moving goalposts and keep the 'real-coverage' label meaningful.
Source: github.com, 2026-11-13
Published there as: “Doc-foundry – AI writes Security detection rules, then proves they work”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.