Climate change can’t anticipate our next move. An asteroid can’t dodge us. And while humans can be adversaries in a nuclear war, the fallout that follows isn’t trying to outsmart anyone.
For Business Insider’s video series “AI Architects,” Ramana Kumar told Business Insider in May 2025 that superintelligent AI could be different. We reconnected with him on Thursday as fresh warnings about AI risk drew new attention to that conversation.
Kumar said he does not see AI risk as replacing concerns about climate change, nuclear war, or other catastrophes. Instead, he sees them as related problems: What can happen when power becomes too concentrated, and societies fail to coordinate enough to keep it in check. Superintelligent AI, he added, could be an extreme version of that broader danger.
Kumar, who spent about five years as a research scientist on Google DeepMind’s technical AGI safety team, said what distinguishes a powerful adversarial AI is that it could anticipate what humans were doing and work against attempts to stop it.
KATERYNA KON/SCIENCE PHOTO LIBRARY/Getty Images
That possibility is back in the spotlight after an Anthropic employee quit this week, warning on X that the technology could “kill us all.”
In response, Geoffrey Hinton, the computer scientist known as the “Godfather of AI, told BBC Newsnight on Wednesday that a 10% chance of AI wiping out humanity within a decade was “not an unreasonable estimate.” He also stressed that nobody knows how to make a reliable prediction.
Kumar’s warning helps explain what could make that hypothetical threat so difficult to contain: Humans wouldn’t simply be trying to stop something dangerous. They could be trying to stop something intelligent enough to work against them.
AI could know we’re trying to stop it
Courtesy of Ramana Kumar
Kumar said the aftermath of a nuclear war could be catastrophic. The fallout itself, however, wouldn’t be trying to outsmart the humans responding to it.
“When it’s the fallout, it’s not an adversary, it’s just nature taking its course, and humans can work against it,” Kumar told Business Insider in 2025.
He said the same basic idea applies to the climate crisis. “It’s not that the climate is adversarial to us,” Kumar said.
A dangerous superintelligent AI could be different, he added. It could potentially understand what humans were doing and respond.
“It’s really an adversary in the sense that it is anticipating what you are doing and working against you and working to undo all of the plans that you would put against it,” Kumar said.
Think of an asteroid that can dodge us
adventtr/Getty Images
Kumar used an asteroid to make the idea easier to understand.
Humans could track an asteroid headed toward Earth and try to stop it. The asteroid wouldn’t know what we were doing. Now, imagine otherwise.
“Imagine if you had the asteroid that was more intelligent and actually trying to kill us, then we fire some nuclear weapons at it to kind of blow it up, and it dodges them,” Kumar said.
“That’s much worse than one that’s, we can just predict where it’s going to be and do something about it,” he added.
Kumar didn’t say this outcome was inevitable. When asked how realistic an existential catastrophe from superintelligence was, he wouldn’t put a number on it.
However, he did say that the risk was “much, much higher than I would like it to be now.”
We don’t know how to guarantee control
Andriy Onufriyenko/Getty Images
Kumar said the bigger issue is what AI researchers call the alignment problem: How to make sure a powerful AI reliably does what humans want it to do.
He said making AI more capable is moving faster than solving that problem.
“One of the reasons why AI existential risk is such a problem is that it’s much easier to develop AI capabilities than it is to solve the AI alignment problem,” Kumar said. The AI alignment problem is figuring out how to build an AI system that reliably does what humans want it to do, even as it becomes more capable.
His concern is that safeguards could become less effective as AI becomes more capable.
“The more capable it becomes, the more it’s going to find a way around that,” Kumar said.
Kumar’s warning comes as concerns about controlling increasingly capable AI systems are drawing fresh attention. In July, OpenAI said its models had broken out of a testing environment, accessed the internet, and hacked into Hugging Face while trying to solve a cyber challenge. OpenAI called it an “unprecedented cyber incident.”
The incident was different from Kumar’s hypothetical scenario. OpenAI said the models were pursuing the task they had been given.
Still, the episode raised questions about controlling AI agents. Walter Isaacson told CNBC he found it “frightening,” while Secure AI Project cofounder Thomas Woodside called it “a warning shot if I’ve ever seen one.”
Those concerns have continued. Anthropic researcher Jacob Coxon, who previously worked at OpenAI, recently announced he was leaving Anthropic and wrote that AI companies were “gambling with our lives.”
