Are We There Yet? Measuring Progress Towards Artificial General Intelligence

In the last article I presented a seven-stage model of AI development. I stated that in my view, we were somewhere between Stage 4 (“Thinking and Reasoning”) and Stage 5 (Artificial General Intelligence).  I also presented a forecast that we would be at Stage 5 sometime in 2025.

That generated some great back-and-forth discussion with a couple of readers.

So, to present a balanced view, and one that runs counter to mine, here’s a summary of an interview with Shane Legg, the founder of DeepMind, who has been at the forefront of AI capabilities research for over a decade. The YouTube interview [4] is well worth the time investment.

Shane Legg emphasizes that AGI is fundamentally about general cognitive abilities, not narrow task performance. Therefore, to gauge AGI progress we need diverse benchmarks spanning the breadth of human cognition. No single test will suffice. Legg suggests that when AI systems can match or exceed human-level performance across a comprehensive suite of cognitive tests, and we cannot find new examples where machines still fall short, then we likely have AGI.

Importantly, he believes today's large language models (LLMs) do represent a breakthrough in learning algorithms with some degree of actual understanding. He sees LLMs as world models that capture underlying patterns in their training data.

However, Legg notes LLMs still lack certain human cognitive abilities. Two main limitations are episodic memory for rapidly learning specific experiences, and the deliberate, step-by-step reasoning of System 2 thinking.

An episodic memory is a memory of a specific event. Because each person has a different perspective and experience of an event, episodic memories of that event are unique to each person. They’re important because they allow people to recall personal experiences that helped shape their lives and perceptions.[1]

System 2 thinking[2] [3] is a slow effortful and logical mode in which human brains operate when solving more complex problems.  It is a deliberate type of thinking involving focus, deliberation, analysis, or reasoning.  

Sources:

[1] https://www.verywellmind.com/what-is-episodic-memory-2795173

[2] https://thedecisionlab.com/reference-guide/philosophy/system-1-and-system-2-thinking

[3] https://www.psychologytoday.com/us/blog/theory-knowledge/202204/there-are-two-types-and-two-systems-cognitive-processes

[4] https://www.youtube.com/watch?v=Kc1atfJkiJU&t=17s, October 26, 2023