GoKawiilThe entirety of Hacker News, stored in parquet files, is approximately 10GB in size. I was interested in analyzing the dataset and, in the fashion of the contemporary zeitgeist, in doing so with Codex. With Modolap, Codex can analyze it well.
After simply adding the skill with npx , the first topic of interest was mention history: whether mentions of Rust superseded those of Go, and MySQL versus Postgres. Simply running
codex "With Modolap, Write a query to analyze historical keyword-based topic mentions over hacker news's history (Dataset's homepage: https://huggingface.co/datasets/open-index/hacker-news/tree/main). Initially, of Rust vs Go."
and some minimal back-and-forth yielded an adequate script using Modolap.
Rust vs Golang
Codex vs Claude Code
Postgres vs MySQL
An additional hypothesis is whether the average comment got shorter. From an initial look, it seems as if there does exist a gradual decline in length.
P50 & Average Comment Length (Chars)