GoKawiil← petervijeh.com I had Gemini train its own replacement for $9
This article was written with the assistance of AI. If that bothers you, stop reading here. The numbers are real: every score comes from the ten training runs described below, and the full run log is in the linked knife.day write-up. The knife.day write-up, with the full run log
I like to cook, and somewhere along the way that turned into an obsession with high-end chef's knives. So I scrape the Reddit threads where people argue about them and pull out every brand, model and steel they mention, to see what is getting bought and argued about.
Picking product names out of text is a job called named-entity recognition, and small models have done it for a decade. I was doing it with Gemini 3.1 Pro, one paid API call per comment. Overkill, but it worked: from "picked up a Mazaki in white #2, way better than my old Fibrox" it returned Mazaki as a brand, Fibrox as a model and white #2 as a steel, and nothing else. But the scraper pulls every new comment, so the bill grew with how much people posted, and the only way to cap it was to skip comments.
The obvious replacement, an open NER model called GLiNER run zero-shot, cut the cost to nothing and the accuracy to about 0.65 F1 against Gemini's answers. That gap is what the rest of this is about: could Gemini label 4,290 comments once and teach GLiNER to close it?
What : Fine-tuned GLiNER large v2.5 (459M) to tag brands, models and materials in Reddit comments, on labels Gemini 3.1 Pro wrote once.
Fine-tuned GLiNER large v2.5 (459M) to tag brands, models and materials in Reddit comments, on labels Gemini 3.1 Pro wrote once. Why : Zero-shot GLiNER scored about 0.65 F1 (est.). Gemini scored well and billed every comment for as long as the scraper ran.
Zero-shot GLiNER scored about 0.65 F1 (est.). Gemini scored well and billed every comment for as long as the scraper ran. Approach : Ask Gemini for strings, not offsets. Compute offsets in code. Add comments with no products in them as negatives. Lock a 225-comment validation set before the second run.
Ask Gemini for strings, not offsets. Compute offsets in code. Add comments with no products in them as negatives. Lock a 225-comment validation set before the second run. Problems : Five of ten runs produced no usable model. Three failed on configuration. Two failed on a tensor called words_mask that I filled the way you fill an attention mask.
Five of ten runs produced no usable model. Three failed on configuration. Two failed on a tensor called words_mask that I filled the way you fill an attention mask. Result : 0.83 F1 against Gemini's labels after 24 minutes on a Tesla T4. $9 of labels, about $2.50 of GPU time, and days of debugging.
... continue reading