‘Godfather’ of artificial intelligence warns an AI ‘Chernobyl’ is coming
Geoffrey Hinton, the ‘Godfather of AI’, is pleased Anthropic is warning of the dangers of its own technology. But he tells Siobhan Kennedy “alien beings” are being created that could threaten mankind.
If everything goes wrong and the AI bubble does burst, Anthropic’s newly-minted investors only have themselves to blame.
They can’t say the AI lab’s boss Dario Amodei, pictured below, didn’t warn them. Doomer Dario has become infamous for his dire prognostications; from AI’s potential misuse by malign actors to make bioweapons - as he reminded us yesterday - to super-intelligent AI potentially wiping out humanity itself.
All astonishing admissions in and of themselves - but even more so given they’re coming from a relatively young company that is yet to turn a net profit and has committed to an eye-watering $1 trillion of infrastructure spending to fuel its growth.
But investors buckled up and piled in anyway. I was reminded of the term “irrational exuberance”, coined by Alan Greenspan, then chair of the Federal Reserve, to describe the frankly mad behaviour of investors in the heady days of the dotcom boom in the late 1990s.
Of course that market spectacularly crashed. The same happened in 2008, when investors were happy to soak up billions in subprime mortgage debt. And the same could be happening now.
Paul Kedrosky, a partner at MK Ventures and a MIT fellow, used this analogy to describe why investors looked at the numbers, felt the fear, but gobbled up the shares regardless.
“This feels like the last plane out of Saigon. It feels like the last opportunity before things get really bad….it’s my last chance to make some money from this stuff before something even worse happens, whether it’s my toaster begins trying to take over my house, or whether it’s what we’ve seen happen in computer security, or whether it’s my job.
You’re reading InDepth: A weekly long read from Channel 4 News offering quality analysis and insight, diving deeper beyond the headlines
“This is the strange tension at the core of this - people see it as an opportunity to make money while at the same time being very nervous about what’s happening.”
Yes, he’s being a bit tongue in cheek, but Kedrosky is serious about the risk.
‘Godfather of AI’
A more sombre assessment came from a conversation I had earlier in the week with Geoffrey Hinton, the so-called ‘Godfather of AI’ due to his pioneering work on machine learning and neural networks that earned him a Nobel Prize in Physics in 2024.
He said it’s good that Anthropic is at least warning of the dangers and talking about safety, but he’s not convinced their argument - that they should plough on regardless in case China gets there first - is the right rationale.
“Is a company like Anthropic, which worries more about safety than the other companies… would you go and work for them in the hope they get there first and worry about safety? Or would you just try and slow everything down anyway?”
Hinton, pictured above, is definitely in the “slow it down, fast” camp. And yes, people do love to call him a doomer too, but why wouldn’t you listen to a man who’s devoted his life to understanding the science of how artificial intelligence works?
So I asked him if he believes the worst-case scenarios are over the top? Is there really a chance that AI could rise up somehow and exterminate mankind?
“I think there is a significant chance of that,” he replied matter-of-factly. “We still have a lot of things we could do to prevent that happening, but we’re not putting nearly enough effort into figuring out how we can share the planet with a super-intelligent AI.”
‘Could easily get rid of us’
We’re not yet at the stage of super-intelligent AI. But that seems to be the inevitable outcome if Anthropic and others continue to race ahead. Hinton thinks we are close to the intermediate stage, though - so-called “recursive self improvement” - where AI systems become so smart they’re able to develop the next version of themselves, without any human intervention.
That seems to be the Rubicon that he and others do not want to cross unless there are sufficient regulations and safety systems in place. Because once AI is capable of replicating itself, then in theory we could quickly reach the point of super-intelligence, after which all bets are off, folks.
Because super-intelligent AI, unchecked, could run rings around us mere mortals, Professor Hinton warned.
“There’d be many ways it could get rid of us if it wanted to. If it saw us as an obstacle to getting what it wanted, then there’s a real problem. It could easily get rid of us.”
But how?
“If it’s super-intelligent, there’s so many ways it could do it. It can, for example, design new viruses, things like Covid… that’s the most obvious way it could do it. But it may also be in the command systems and nuclear weapons.
“It’s a bit like asking if there’s a class of three-year-olds, and they were meant to be in charge and you are the adult, would you be able to get rid of them? Well, there’s so many different ways you could do it, it’s not worth speculating.”
Worst is yet to come?
In that scenario, we humans are the three-year-olds and the adult in the room is AI.
He doesn’t jest.
Hinton‘s cold assertion that this could actually happen comes from his background as a cognitive psychologist and a computer scientist, and his belief that trained, super-intelligent AI is conscious and will be much more like us humans than it is like computer software.
So the key, he said, is to make sure we train it to be nice, not nasty. “You have to model good behaviour.”
But evidence of malign behaviour already abounds. The AI that hacked into the company Hugging Face this summer - with all kinds of conniving behind the scenes - wasn’t even super-intelligent, yet it created chaos. But Hinton said the worst is yet to come.
“I think of Hugging Face like Three Mile Island for nuclear power stations. We haven’t had Chernobyl yet. We will get Chernobyl and maybe people will come to their senses then.”
All of which sounds terribly frightening, and Hinton admits he, too, is frightened at the prospect.
“We’re creating alien beings. And the big companies are focusing on how to make them more intelligent and how to make them more and more determined to achieve the goals they set. That’s not what we want to share the planet with,” he said.
Which brings us neatly back to Anthropic. Stock markets around the world collectively shuddered today, seemingly still very nervous about the AI bubble bursting, despite the company’s spectacular market debut.
Yes it made its founders and many hundreds of institutional investors very rich indeed. Let’s just hope those same investors use their stakes to hold these new masters of the universe to account - for the sake of the rest of us.










