Don’t be fooled—LLMs don’t reason
Original reporting by MIT Technology Review
AlphaGo is an artificial intelligence program developed by DeepMind that famously defeated Go world champion Lee Sedol in 2016. During their historic match, AlphaGo made a seemingly nonsensical move that baffled human commentators, yet proved strategically brilliant, leading many to hail it as a flash of machine creativity or intuition. However, as one of AlphaGo's core developers explains, this pivotal moment was not a product of gut feeling, but of profound machine reasoning—a capability critically absent from today’s most advanced AI systems.
A fundamental difference
Unlike Deep Blue's brute-force chess calculations, AlphaGo mastered the far more complex game of Go by employing a dual-system approach. It combined an "intuitive" neural network, trained to mimic expert human plays, with a powerful search mechanism that explicitly weighed the future consequences of potential moves, selecting options no human would consider. This sophisticated deliberation contrasts sharply with current large language models (LLMs), which operate primarily through fast, associative pattern completion. While LLMs excel at fluency and even simulate "chain of thought," their outputs stem from iterative next-token prediction, lacking an auditable, evolving epistemic state or a clear separation between knowledge and reasoning. This fundamental difference means that despite their impressive scale, LLMs cannot provide the verifiable, trustworthy insights required for high-stakes fields like science and medicine. The author argues for a return to AlphaGo’s architectural principles, advocating for AI systems that build knowledge through transparent, evidence-backed deliberation, enabling truly novel discoveries.
The AlphaGo match in 2016 offered a crucial insight: genuine machine intelligence, capable of truly creative and trustworthy insights, emerges from a powerful synthesis of intuition and rigorous, explicit reasoning. Unlike the associative pattern completion that defines contemporary large language models, AlphaGo’s architecture—with its distinct policy network and deliberative search mechanism—demonstrated a machine analogue of human System 1 and System 2 thinking. This fundamental distinction underscores why simply scaling current AI models, while enhancing their "intuition" and fluency, will not yield the auditable, verifiable breakthroughs required for high-stakes domains. Achieving reliable knowledge generation necessitates equipping future AI with explicit epistemic states and transparent, step-by-step belief revision, allowing them to truly "hold a position and weigh possible futures."
Towards Auditable Breakthroughs The implications of embracing this architectural paradigm are profound for the future of AI and its societal impact. By developing systems with robust reasoning capabilities that parallel the scientific method, we can create AI that not only proposes solutions but rigorously justifies its conclusions through an inspectable chain of evidence and inference. This shift is critical for fields like drug discovery, materials science, climate modeling, and medical diagnosis, where the stakes are immense and errors demand clear accountability. Such reasoning-driven AI promises to unlock genuinely novel insights, moving beyond human-imitable patterns to generate groundbreaking, reliable knowledge. This approach cultivates trust and transparency, shaping a future where AI acts as an indispensable, verifiable partner in addressing humanity’s most complex and urgent challenges.
Frequently asked questions
- Why was AlphaGo's Move 37 against Lee Sedol considered so revolutionary in AI?
- Move 37 was initially perceived as a mistake but led to victory, demonstrating AlphaGo's profound capabilities. It combined an "intuitive" neural network with a "deliberative" search mechanism, allowing it to explore countless future possibilities and make strategically complex moves that surprised even grandmasters. This showcased a form of machine creativity and reasoning beyond brute-force calculation, contrasting sharply with previous AI approaches that relied solely on extensive pre-programmed rules.
- How does AlphaGo's reasoning differ from current large language models like ChatGPT?
- AlphaGo utilized two distinct modes: an intuitive "policy network" and a deliberative "search machinery" that explicitly weighed future consequences. Modern large language models primarily operate on a single "next-token prediction" process, akin to fast pattern completion. While LLMs can generate "chains of thought," these are often produced by the same underlying associative process, lacking AlphaGo's separate, auditable reasoning mechanism and explicit knowledge state about the problem.
- What kind of reasoning capabilities are needed for future AI in scientific discovery or medicine?
- Future AI in critical fields needs genuine reasoning to produce trustworthy and novel insights. This involves systems that maintain an explicit, inspectable record of what they know and doubt, clearly separate knowledge from its manipulation, and construct conclusions through an auditable sequence of evidence and inference. This approach helps pinpoint errors and ensures reliability in high-stakes applications like medical diagnosis or drug discovery, moving beyond mere associative pattern recognition.