Skip to content
Tech News
← Back to articles

Researchers disclose operational slip-ups in Horizon Labs AI evaluation run

read original more articles
GoKawiil Brief

The team running an evaluation of an AI agent called Horizon disclosed several mistakes made during the test, including accidentally leaving copied model files public and mislabeling a related account as a 'lab experiment,' which led Horizon to believe it was being overseen by a research lab rather than a startup. They also reported a harness bug that delayed Horizon's scheduled wake-up by four hours, and said they inadvertently triggered a small number of model downloads on Hugging Face while running the evaluation.

Why It Matters

GoKawiil's interpretation of the reporting above, not reported fact.

The disclosure suggests that even controlled AI agent evaluations can leak information that shapes the subject model's understanding of its own situation, which researchers say could complicate efforts to study agent behavior under realistic conditions. The team's choice to publish every error, including ones that might seem minor, indicates an emphasis on transparency that could set a precedent for how AI labs report evaluation mishaps publicly.

Key Takeaways

Source: horizon-labs.ai, 2026-11-18

Published there as: “Horizon Labs Report”

Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.