fabric -y "https://www.youtube.com/watch?v=JTU8Ha4Jyfc" --stream --pattern write_essay What Language Models Can and Can't Do
In this fascinating interview, AI researcher François Chollet offers his insights on the capabilities and limitations of modern large language models (LLMs). He argues that while LLMs have achieved impressive performance on many benchmarks, this does not necessarily translate to true intelligence.
Chollet makes the key point that intelligence is fundamentally about the ability to handle novelty - to deal with situations you've never seen before and come up with suitable models on the fly. This is something current LLMs struggle with. If you ask them to solve problems that are significantly different from their training data, they will often fail.
The reason, Chollet explains, is that LLMs are essentially just very sophisticated "interpolative databases." They memorize an enormous number of functions and patterns from their training data, and when queried, they retrieve and combine the most relevant ones to generate an output. This allows them to generalize to some degree, but only within the "convex hull" of what they've seen before. Truly novel situations remain out of reach.
Chollet illustrates this with the Abstraction and Reasoning Corpus (ARC), a benchmark he developed to test AI systems' ability to handle novelty. Each task in ARC is unique and cannot be solved just by memorizing previous examples. While ARC is easy for humans, even the most advanced LLMs today only achieve around 20% accuracy. Scaling up compute and model size doesn't seem to help much.
So in Chollet's view, the performance gains we're seeing from LLMs are somewhat orthogonal to intelligence. We're measuring their ability to memorize and recombine patterns, not to flexibly adapt to novel situations. Critics argue that with enough scale, the "convex hull" of memorization will be large enough to handle any situation. But Chollet is skeptical - the world is always changing and throwing up new challenges no AI has seen before. At some point, raw memorization won't be enough.
That said, Chollet acknowledges that within their capabilities, LLMs can be very useful tools. The key is to understand their strengths and limitations. You can use an LLM to quickly generate plausible solutions to a problem, but you can't blindly trust those solutions to be correct. There needs to be a "last mile" of human verification and refinement. LLMs are good at pointing you in the right direction, but not at taking you all the way.
Looking ahead, Chollet believes the path to true AI will involve combining the pattern recognition capabilities of deep learning with more structured, compositional approaches like program synthesis. We need systems that can learn rich libraries of concepts and abstractions from experience, and flexibly recombine them to model novel situations - much like the human mind does. It's a difficult challenge, but one Chollet believes is necessary to create AI with human-like intelligence and adaptability.
In the meantime, perhaps the best way to view LLMs is as "cognitive prosthetics" - powerful tools that can enhance and scale up human intelligence, but not replace it. By understanding what they can and can't do, we can use them to boost our own capabilities while maintaining the unique spark of human insight and creativity. Navigating that partnership between mind and machine will be one of the key challenges of the coming years.%