Imagine you have to generate the word that succeeds this colon: ___
What would you put in that blank space?
It’s easier when the question comes first.
But what if we structured things such that the blank had to be generated before its constituent parts. LLMs are wonderful, but I see too many people try to break down recursively to solve problems like top-down humans do. Instead, I posit that FORTH and associative/applicative languages may be better for transformer architectures. Concatenate, not integrate. Agree on the stack state.
I set out to question if this could be true.
Sideways Passing Join.
Imagine you had this program:
A SCAN [foo > 5] FILTER B SCAN [foo < 5] FILTER BUILD PROBE
that performs a natural join on A and B’s shared identifiers.
Because of the properties of associative languages you can always make local edits. For example, if you made a sed-like transformation, you could replace BUILD PROBE with the following anywhere there’s a BUILD PROBE sequence to do a sideways-information-passing join:
... continue reading