AIThe Decoder1h ago
The inventor of the bitter lesson has a bitter lesson for LLM scaling
The inventor of the bitter lesson has a bitter lesson for LLM scaling enthusiasts
TL;DRScaling compute alone won't solve AI; fundamentally new methods are needed.
Why it matters: Challenges the dominant strategy of major AI labs, suggesting current approaches may hit diminishing returns without algorithmic breakthroughs.
Turing Award winner Richard Sutton reaffirms that raw compute alone isn't enough for intelligence and that fundamentally new learning methods are needed. Ex-OpenAI researcher Tworek agrees, saying blind transformer scaling "has reward hacked brains of a whole generation of…
Read full articleSource: The Decoder · Opens in new tab