Agents as Sparks of AGI and Turbochargers of Existing AI Model Benchmarks

Agents may be one small step for AI, but they may be one giant leap for (or against) humankind.

There are various definitions of artificial general intelligence (AGI). These include the ability to do all jobs humans can do, to do anything humans can do, to hypothesize and conduct science experiments, to demonstrate embodied intelligence, and to demonstrate sentience and consciousness. The more a definition demands, the farther we are from it.

But as more and more people recognize (Altman, Kurzweil), AGI isn’t a switch or a line we will cross; instead, it is something we will move closer and closer toward, with continued advances putting us closer to AGI, and those advances will have meaningful consequences.

In a presentation last week, Andre Ng said agents are a small step towards AGI because they can plan, reflect, reason, debate, and correct themselves before producing output. This allows them to exceed existing model benchmarks (he cites HumanEval) of current models. Even when agents are used with models such as ChatGPT3.5, these AIs can achieve greater than a 94% score, blowing past ChatGPT4 and other similar models.

Professor Ng says this is a small step on the path toward AGI. Why? I’m not 100% sure, as he doesn’t say, but consider the following.

(1) Define AGI as being what all humans can do.

(2) Imagine individual agents only as AIs that can produce quality output that is at least as good as what humans can produce when given a task or set of tasks.

(3) Imagine swarms of agents working together both to accomplish the task and to conduct multiple tasks.

It is not hard to see how these could lay the foundations for AGI.

One thing that fellow Stanford professor Erik Brynjolfsson often points out is that today’s AIs can mostly do tasks and not entire jobs, but if AIs agents can now perform these tasks reliably and work with swarms of other agents, then they can potentially do jobs.

This is potentially what leads Andrej Karpathy to conclude that we can expect AGI to develop within “nooks and crannies” of the economy.

Does Karpathy give any examples? No. But recently Nvidia has been suggesting it can make AI nurses available and for only $9/hour.

Is this actually AGI even under the most minimalist definition (it can do all of our jobs)? No. But as agent-supported models start doing all of our jobs one task and then one job at a time, we have a clear path towards AGI, one small step at a time.

Agents may be one small step for AI, but they may be one giant leap for (or against) humankind.

As computers reach human-level intelligence in more and more “nooks and crannies” of the economy, how will education adapt?

I wrote more on agents here.

For even more, check out our report and our book.