“The Bitter Lesson” is wrong. Well… sort of. Assaf Pinhasi 3 min read · 1 hour ago 1 hour ago -- Listen Share
TL;DR
There is no dichotomy between domain knowledge vs. “general purpose methods that leverage data+compute”.
They are both powerful tools that compensate for each other and need to be balanced and traded off during the model building process.
“The bitter lesson” in 30 seconds
“The bitter lesson” is one of the most popular opinion pieces about AI research and it’s future.
In his writing, Rich Sutton makes a dichotomy between two main schools of AI research:
1. Research based on “human knowledge”
2. Research based on scaling methods like “learning and search” that scale with more data and compute.
Sutton claims that virtually all long term progress in AI was achieved using the latter, and that the former is actually detrimental and distracting.
... continue reading