GoKawiilKernel.org Says AI Scrapers Now Outweigh Legitimate Git Traffic in Server Load
The maintainer of git.kernel.org reports that AI training crawlers are consuming more CPU resources than all legitimate access combined, including actual git clones. Across five geographically distributed nodes, 14 CPU cores are constantly occupied just rendering git commits as HTML pages for these bots, even though the same data is available far more efficiently via direct git clone.
GoKawiil's interpretation of the reporting above, not reported fact.
This highlights a growing infrastructure burden that open-source projects face as AI companies scrape publicly available code and history to train large language models, often in the least efficient way possible. The inefficiency wastes real computing resources and money for community-run services, raising questions about whether AI firms will ever adopt more considerate scraping practices or compensate the infrastructure they depend on.
- AI crawlers now use more CPU than legitimate git access on kernel.org's infrastructure.
- Kernel data is valuable to AI trainers because it's guaranteed free of AI-generated content.
- Crawlers inefficiently render web pages instead of using efficient git clone methods, wasting resources.
Linux Kernel Development by Robert Love — If reading about git.kernel.org's commit history makes you curious what all those patches actually do, Robert Love's classic is the friendliest way in. It walks through the kernel's subsystems in plain language, so the next time you browse a commit log you'll know what you're looking at — pure pre-AI, human-written knowledge.
See Linux Kernel Development by Robert Love on Amazon → Affiliate link — we may earn a commission on purchases, at no extra cost to you. Product picked by AI based on this article; it is not a tested recommendation.Source: people.kernel.org — Konstantin Ryabitsev, 2026-08-29
Published there as: “Creepy Crawlies”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.