GoKawiilMemdoor-built multi-agent harness matches GVS5H's Qwen3.8-27B system on LiveCodeBench Hard subset
A reproduction built by Memdoor's coding tool, working only from a written description of the GVS5H paper's method, replicated the five-role multi-agent workflow using Qwen3.8-27B models. Across five passes on the first ten problems from the authors' LiveCodeBench Hard list, the rebuilt workflow averaged 9.8 of 10 correct, versus 9.6 for the authors' own harness and 8.0 for a single Fable 5 model, with full logs and costs published for all 50 runs.
GoKawiil's interpretation of the reporting above, not reported fact.
The result suggests a multi-agent prompting structure, rather than a specific proprietary implementation, may be what drives the performance gain over a single large model on coding benchmarks, since an automatically generated reimplementation matched the original authors' numbers. However, the test used only ten problems rather than the authors' full 100, and the publishers themselves note that because the contest problems predate the model's possible training cutoff, contamination of results cannot be ruled out.
- A reconstructed five-agent Qwen3.8-27B workflow scored 9.8/10 average versus 9.6 for the original authors' harness and 8.0 for a single Fable 5 model.
- The reproduction was built by an AI coding tool directly from a description of the method, without human-written code.
- Full logs, costs (~$35.79 for 50 runs) and caveats about possible training contamination were published alongside the results.
Source: github.com, 2026-10-31
Published there as: “Memdoor: A team of 5 Qwen3.8-27B models beats Fable 5 on LiveCodeBench Hard”
Read the original report → The summary and analysis above are GoKawiil's own, written from reporting by the source above. Facts and quotes belong to the original publisher.