GoKawiilWe’re introducing Pulpie, a family of Pareto-optimal models for extracting main content from HTML pages. Pulpie approaches SOTA extraction quality at one twentieth the cost.
Our smallest model, pulpie-orange-small , scores 0.862 ROUGE-5 F1 on WebMainBench. This matches Dripper, the leading extractor, which scores 0.864. Pulpie’s performance is despite it being a third the size: 210M parameters versus Dripper’s 600M.
The gains come from architecture. Pulpie is an encoder that labels every HTML block as content or boilerplate in a single forward pass. This also makes it fast.
On an NVIDIA L4 GPU, pulpie-orange-small processes 13.7 pages/sec against Dripper’s 0.68 pages/sec. At $0.39/hr for an L4 instance, cleaning 1 billion pages costs $7,900 with Pulpie and $159,000 with Dripper.
Pulpie unlocks high quality web extraction at a scale impossible before. We expect this to benefit pre-training and context management.
Our models are open source and available on Hugging Face. See Get started for instructions.
Extraction is the bottleneck
Language models consume the web twice. First in pre-training, where they learn about the world. Then at inference, when they pull in relevant context. Both times the input is mostly noise. During discovery, we found 70% of the blocks on a typical HTML page hold boilerplate like navigation, ads, sidebars, and footers. Main content is only a small fraction of the page.
However, that fraction determines model quality on both ends.
AICC (Ma et al., 2025) measured the effect of cleaner extraction on pre-training. The team built two corpora from the same Common Crawl snapshot. One extracted content with heuristics. The other extracted it with a model-based parser. Everything else in the data pipeline remained equal. They then trained an identical model on each corpus.
... continue reading