AGI remains one of the biggest milestones the AI industry likes to invoke. There is still no consensus on exactly how far AI must advance before it can be called AGI, but leading figures across the industry continue to describe that point according to their own criteria. During NVIDIA’s earnings call on August 26, a question about AGI came up, and Jensen Huang’s answer was relatively unequivocal.
“In a lot of ways, and for many tasks, we could say that we’ve already achieved AGI.”
But he immediately downplayed the weight of that declaration.
“I think all of those milestones … they’re kind of senseless at this point.”
What Huang emphasized next was not the definition of AGI or any particular benchmark. He argued that what matters more is whether AI is doing productive and useful work in practice, whether it is generating profitable outcomes, and how much its capabilities can scale when given more compute. He also stressed that AI is moving beyond models that simply respond to prompts toward agentic systems that can carry out multiple steps on their own and continue deciding what to do next.
In other words, after answering the question of whether AGI had been reached, Huang quickly shifted the focus of evaluation elsewhere. Rather than asking what label can be attached to AI, he was arguing that we should look at what it can actually accomplish. Of course, declaring AGI without a clear definition remains open to debate. What stands out in Huang’s remarks, however, is that attention is moving away from the label of AGI itself and toward what AI can actually do in real-world environments.
Evaluation methods are changing accordingly. Increasingly, researchers are looking beyond a model’s standalone score and examining the broader system around it—what tools and memory it is connected to, and how it is allowed to act.
The Same Model Can Perform Differently Depending on the System
Recent results from ARC-AGI-3 illustrate why it has become difficult to explain an AI system’s actual capability through model performance alone.
ARC Prize is an AGI benchmarking project designed to evaluate whether AI can discover new rules and adapt to unfamiliar problems. Its recently released ARC-AGI-3 goes a step beyond static problem-solving by placing AI inside interactive environments where it must act directly, observe the consequences, and work out the objective for itself.
On July 24, ARC Prize validated Claude Opus 5 at 30.16% on the ARC-AGI-3 public set. Less than a month later, on August 21, NVIDIA reported a score of 100.00 using the same model on the same public set. The model itself had not changed. Its weights had not been retrained, nor had it been replaced with a larger model. What changed was the harness surrounding it.
A harness is the agent system that determines what information the model retains, what it tries first, how it approaches a problem again after failure, and how many actions it is allowed to take on a task. Put simply, if the model provides the ability to “think,” the harness determines how that ability is translated into actual behavior.
This distinction matters because of how ARC-AGI-3 is evaluated. It is not a test in which the AI simply reads a question and selects one correct answer. The AI enters an interactive environment without being told the rules, acts within it, observes the results, and figures out what it is supposed to do. Exploration, revision, and repeated decision-making matter more than a single response.
That means the result can change dramatically depending on what information is preserved, how previous failures are used, and how the next action is chosen. At the same time, NVIDIA’s score of 100 should not be treated as an absolute improvement in the model’s own capability. The result came from the public set, and the same level of performance has not yet been validated on the private set. The researchers themselves also explicitly acknowledged this limitation.
Even so, the implication of the case is clear.
The score was produced not by the model alone, but by the entire system surrounding it.
Games Have Long Designed AI as a “System”
Game AI has long focused less on building a single general-purpose intelligence and more on defining what is needed first—when an enemy should hide or attack, how an NPC should choose an action, or how the intensity of combat should be adjusted—and then designing a system around that role.
In the 2001 science-fiction first-person shooter 'Halo', AI-controlled enemies take cover, move to flank the player, and react to the combat situation. In the 2005 horror-action FPS 'F.E.A.R.', enemies change how they move and attack according to the situation and their goals, creating the impression of coordinated behavior. In the 2000 life simulation game 'The Sims', characters choose actions according to changing needs such as hunger, fatigue, and hygiene. They do not behave this way because they think like humans, but because their behavioral rules and decision structures were designed around the roles each game required them to perform.
One of the clearest examples is the 2008 cooperative zombie FPS 'Left 4 Dead'. The game features a system literally called the “AI Director.” Rather than simply making individual zombies more intelligent, it is tasked with shaping the rhythm of the overall experience by adjusting enemy appearances, item placement, and combat intensity according to the player’s condition and the flow of battle.
In other words, instead of concentrating intelligence in a single character, the game assigns it a role in managing the flow of the entire experience.
These systems did not need to pass intelligence tests in the conventional sense. What mattered was not how much they could think like humans, but how well they performed the role assigned to them within the game.
Generative AI Expands the Range of Roles
In the early 2020s, the emergence of generative AI began to change the roles game AI could take on. But what changed was the scope and form of those roles, not the underlying principle of designing AI around the place where it is needed.
Where traditional game AI handled relatively well-defined functions such as combat decisions, movement, need selection, and difficulty or intensity adjustment, generative AI added capabilities such as natural-language understanding, memory, contextual interpretation, and planning. It became possible for AI characters to carry on conversations with players, remember previous situations, and decide what to do next according to the surrounding context. But simply being able to do more does not automatically create a better game experience. As roles become more complex, it becomes even more important to ask where AI’s capabilities and limitations are placed, and how the game’s rules are designed to accommodate them.
The contrast becomes clearer when comparing 'inZOI' and 'MIMESIS'.
In 'inZOI', Smart Zoi is designed to make characters behave more autonomously and create a more natural sense of everyday life. Characters react to their surroundings according to their personalities and life goals, and they can adjust their schedules based on what happened during the day. Players can also view a Smart Zoi’s internal thoughts or influence its behavior and goals through free-form input.
In this kind of structure, what is ultimately expected from the AI is natural, human-like behavior. As a result, actions that fall out of context, repetitive responses, or unpredictable behavior are easily perceived as imperfections. Smart Zoi has in fact experienced issues during live service, including repeated behaviors and periods in which certain functions were temporarily disabled, with related fixes continuing into 2026.
'MIMESIS' places AI’s imperfections in a very different position. In this game, AI imitates the behavior and voices of human players and blends in among them. The player’s task is to suspect and identify who is a real person and who is an imitation. As a result, awkward movement, subtly off responses, or slightly strange timing do not necessarily remain simple defects. They become clues that can reveal the presence of AI. Imperfections that might break immersion in another game instead become material for tension and deduction.
The difference in how AI functions in the two games cannot be explained simply in terms of which AI is more capable. What matters more is what role the AI’s characteristics are assigned to, and what meaning those characteristics are given within the game. In 'inZOI', awkward behavior can surface as a flaw that breaks immersion. In 'MIMESIS', the same awkwardness can become a clue the player is meant to notice.
That is ultimately what generative AI has changed in game design. It has not simply enabled a greater number of actions. It has expanded the range of design choices through which both what AI does well and what it does poorly can be incorporated into the rules and roles of a game.
The Question Game AI Raises
The principle visible in games is not limited to games. Modern AI systems do not operate through the model alone. Search, memory, external tools, and action structures are combined to create an agent system. The large performance gap seen with the same model in ARC-AGI-3 was also tied to differences in this broader system configuration.
This is where the problem game AI has dealt with for years begins to overlap with the wider AI landscape. Games have never necessarily aimed to make every AI as intelligent as possible. Enemies need to apply the right amount of pressure. Companions need to help at the right moment. An AI Director needs to regulate the flow of play. Different roles require different capabilities and different boundaries on what the AI should be allowed to do.
The same is true outside games. Customer-service AI, research assistants, coding agents, and autonomous robots do not require the same capabilities. Even when they use the same underlying model, the criteria for a good system change depending on what job the AI has been given, how much it is allowed to decide on its own, who checks its mistakes, and where it is required to stop.
This distinction becomes even more important with agentic AI. Unlike a model that produces a single response, an agent can continue through multiple steps and may take actual actions in external systems. In that context, high performance alone is not enough. We also need to look at what the AI has been assigned to do, how far the consequences of failure can spread, and how the surrounding system is designed to absorb those failures.
Ultimately, high benchmark scores alone cannot fully explain real-world performance. What matters is how well the model’s capabilities, the role it has been given, and the system surrounding that role fit together.
A good AI may not be the one that can do the most, but the one designed to perform the job it has been given at the level actually required.
The Question That Remains After AGI
Jensen Huang’s emphasis on what AI can actually accomplish, rather than on AGI as a milestone in itself, reads somewhat differently when viewed alongside the cases above. Model performance still matters, but the model’s capabilities alone do not determine the outcome.
In ARC-AGI-3, the same model produced dramatically different results depending on the system it operated within. Games, meanwhile, have long shown that the player experience depends not only on AI capability, but on what role the AI is given and how it is positioned within the system. The rise of generative AI has not fundamentally changed that principle. If anything, as AI becomes capable of doing more, it becomes even more important to consider what tools it can use, how much authority it has, how failure is handled, and how well its capabilities match the role it has been assigned.
Ultimately, building the most capable intelligence and producing the best outcome are not the same problem. The debate over whether AI has reached AGI will continue, but in real-world applications there is another question that may need to be answered first.
Where, and how, should that intelligence be deployed?
