GoKawiilI remember walking to the bus from high school, staring at a Motorola 6800 instruction set manual. I didn’t really understand what I was looking at—boolean expressions, instruction encodings, timing tables—but I was obsessively fascinated by the mechanism of it all. Here was this complicated machine where if I understood it I would have power & control.
I feel the same way about AI models right now. I don’t claim to understand the details, not yet, but I’m fascinated by the mechanism of it all. I’m interested in both:
How models work but also,
The machinery that makes a model.
It’s this latter topic, how a model gets constructed, that I will begin to explore in this post (& possible followups).
Baking
Saturday’s Focaccia
I love baking. You take ingredients in one form & transform them to a totally different form. The ingredients aren’t palatable in themselves but what you create from them is delicious. Oh and also baking is sensitive to initial conditions—you can make a small change early in the process & it will have a large consequence later.
I’ve been experimenting with cold proofing, where you let the yeast do its work overnight in a refrigerator. As I was working to understand models it struck me that there’s an analogy there to creating models, at least as I understand the process so far (please correct me in the comments if I’ve gotten something wrong).
First, though, a progressively revealed story about what we mean by a model. I’ll over-simplify but then reveal more complexity a little at a time.
... continue reading