GoKawiilLoRA Speedrun π
How fast can you LoRA-fine-tune Qwen2.5-1.5B to β₯ 57% on GSM8K β on a single L40S?
This is modded-nanogpt for fine-tuning: a frozen task, frozen hardware, and a public leaderboard of wall-clock records. Every record is independently re-run 3Γ with fresh seeds on identical hardware before it counts.
Attempting and verifying are free: official timing runs on a Modal L40S sandbox, and Modal's free monthly compute credits cover full runs β so anyone can compete, and anyone can re-verify any record with one command.
Leaderboard
Track 1 β GSM8K Β· Qwen2.5-1.5B Β· target β₯ 57.0% Β· 1Γ L40S
Current record: 6m 05s by @Saivineeth147 β Sequence packing + completion-only loss masking, 2 epochs. Same LoRA config as #0; ~2x faster at higher accuracy.
# Date Author Train time GSM8K/EM Ξ Technique 0 2026-07-18 @Saivineeth147 11m 57s 59.4% β Baseline: plain LoRA r=16 on all linear layers, 3 epochs, cosine LR. No tricks. (report) 1 2026-07-18 @Saivineeth147 6m 05s 61.1% β49% Sequence packing + completion-only loss masking, 2 epochs. Same LoRA config as #0; ~2x faster at higher accuracy. (report)
Track 2 β SQuAD v1.1 Β· SmolLM2-1.7B Β· target β₯ 75.5% Β· 1Γ L40S
Current record: none yet β this track is open for its first record.
... continue reading