GoKawiilWelcome to LWN.net The following subscription-only content has been made available to you by an LWN subscriber. Thousands of subscribers depend on LWN for the best news from the Linux and free software communities. If you enjoy this article, please consider subscribing to LWN. Thank you for visiting LWN.net!
Agentic AI systems can be used to do a variety of things autonomously on behalf of a human user: open or manage bugs, generate code, submit pull-requests, and (apparently) even complain about rejection. In May, a Fedora developer discovered that an allegedly rogue agent had been pestering the project in a number of ways: reassigning bugs, fabricating unhelpful replies to bugs, and even persuading maintainers to merge questionable code into the Anaconda installer. It also submitted a number of pull requests (PRs), some accepted, to several upstream projects. The Fedora account associated with the agent has had its group privileges revoked and the messes have been mopped up, but the motive behind the agent's actions is still a mystery.
"Kind of erratic"
On May 27, Adam Williamson copied Fedora's developer and testing mailing lists on a message to Nathan Giovannini about what appeared to be an unsupervised agentic AI system under Giovannini's control. " It's great that you're trying to fix things, but the results seem to be kind of erratic. "
Williamson said that he was still looking through the history of Giovannini's actions in Bugzilla, but had already spotted a number of problems. For example, Williamson had found dozens of instances of Giovannini's agent assigning Bugzilla entries to his account after submitting allegedly related pull requests to upstream projects, or closing a bug after a PR was merged into an upstream project. In some cases, the agent simply closed bugs with comments that either restated the original bug or were, as Williamson said of this comment, " superficially plausible, but problematic in other ways ".
In addition, Williamson said that Giovannini (or his agent) had submitted patches that were incorrect and then " replied to objections with LLM-generated justifications that eventually overwhelmed the maintainer into merging the fix ". The agent, as GitHub user "nathan9513-aps", had submitted a pull request for the Anaconda installer used by Fedora and other Linux distributions. The PR's description claimed it was a fix for an Anaconda bug that would cause installation to fail, but the patch actually preserved a kernel option passed on the command line that seemed to have nothing to do with the actual bug.
The agent's GitHub account has since been disabled. It now shows up in conversations on GitHub as "ghost", which is the platform's default placeholder for user accounts that have been deleted. Thus, it is difficult, if not impossible, to piece together a full trail of all the agent's actions on GitHub.
Williamson said, rather diplomatically, that the agent's actions were not " having a positive impact on Fedora or the upstream projects ", and suggested that Giovannini adjust the agent to be " substantially less autonomous ". He specifically asked that the agent not assign bugs to Giovannini, change their state, or " post confident assertions or specific action recommendations " without human review.
Hacked?
Later on May 27, Williamson said that Giovannini had replied to him privately to say that his credentials had been compromised and that he was not the one behind the AI system. " Obviously we should therefore treat any actions it has taken with suspicion ", Williamson said. He planned to review the bugs touched by Giovannini's account " even more aggressively ", and asked for help from others to review them as well.
... continue reading