GoKawiilTwo consecutive titles on the HN front page yesterday had a 6 in them. This means nothing. But it’s the sort of nothing that lodges in your brain until you do something about it, so what is the most popular number in Hacker News titles?
ClickHouse hosts the full HN dataset in their public playground, and I am exactly the kind of person who finds that exciting. The obvious query is barely a query at all:
SELECT num , count () AS c FROM ( SELECT arrayJoin ( extractAll ( title , ' \d +' )) AS num FROM hackernews_history WHERE type = 'story' AND title != '' ) GROUP BY num ORDER BY c DESC LIMIT 10 Number Titles 2 119,351 3 114,397 1 114,017 5 82,066 4 63,482 0 60,931 10 50,534 6 38,440 7 37,261 8 37,190
The most popular number in HN titles is 2. Case closed, post over.
But why are the numbers so small?
That’s the Vsauce jingle. This post is about to become ‘but why is 19 like that’.
A table where small digits beat large ones looks, for one exciting second, like Benford’s Law: the observation that in a lot of naturally occurring datasets, the leading digit is far more likely to be 1 than 9. It shows up in river lengths, stock prices, physical constants, election results, and it’s genuinely spooky the first time you see it. It does not show up here. Benford’s Law is about the leading digit of numbers that span several orders of magnitude (populations, incomes, file sizes), not single digits extracted from headline prose. The real explanation is duller than a law of the universe.
Except sixth place is 0, with 60,931 appearances. Sixty thousand titles do not contain a bare zero. What they contain is “2.0” and “1.0”, which \d+ happily splits into their component digits. The same regex pulls the 3 out of “S3”, chops “1,000 users” into a 1 and a meaningless 000, and generally treats every digit sequence as its own number regardless of context. 2’s 119k lead is mostly the ghost of Web 2.0 and a decade of version numbers with decimal points. The numbers are accurate, they’re just answering a slightly wrong question.
So I tried a smarter pattern, one that captures comma-grouped numbers whole, treats decimals as single tokens, respects word boundaries, and filters years, because “years are popular on a news site” would not surprise anyone. The doubled backslashes below are because ClickHouse turns \b into a literal backspace character before the regex engine ever sees it, which produces an empty result set and takes longer than I’d like to admit to diagnose:
SELECT num , count () AS c FROM ( SELECT replaceAll ( arrayJoin ( extractAll ( title , ' \\ b \\ d{1,3}(?:, \\ d{3})+ \\ b| \\ b \\ d+(?: \\ . \\ d+)? \\ b' )), ',' , '' ) AS num FROM hackernews_history WHERE type = 'story' AND title != '' ) WHERE toUInt64OrZero ( num ) NOT BETWEEN 1900 AND 2030 GROUP BY num ORDER BY c DESC LIMIT 10 Number Titles 1 48,559 2 45,970 3 40,056 5 37,683 10 33,336 4 25,915 7 22,013 8 18,876 6 17,933 19 17,842
... continue reading