AI Could Kill Us All... So Let AI Companies Regulate Themselves?
If you’ve been following AI news, you’ve probably seen that former Anthropic employee Jacob Coxon resigned last week because he was “genuinely frightened” over AI being “out of control.”
As part of a longer post on X, Coxon wrote that “the people building AI earnestly believe that it could kill us all by the end of the decade.”
In response, Dario Amodei, the CEO of Anthropic, released a letter titled “We Must Pace the Frontier,” where he doesn’t refute anything that Coxon says.
Amodei actually seems kind of on the same page as Coxon, and he outlines this three-pronged plan to slow the pace of AI.
But here’s where, to me, it goes off the rails.
In Amodei’s plan, he says that the “frontier AI companies should work together” to set the AI law standards, and that the government should “enable these discussions, but they don’t need to participate.”
Amodei wants these billion-dollar AI megacorps to set their own safety standards while we wait years for actual laws to pass. Are you kidding me?
That’s like if John Hammond, the head of InGen, the company that cloned Dinosaurs in Jurassic Park, wrote an open letter saying:
“Guys, look, there’s a good chance the dinosaurs we’ve been breeding will kill us all by the end of the decade... but I’ve got a plan.”
The rest of the world’s like, “Yeah, we’re listening.”
And John goes: “Okay, hear me out. What if we at InGen, along with the other dinosaur companies, write our own dinosaur containment laws while we continue to breed velociraptors and make billions of dollars? The government doesn’t even have to get involved, at least not yet! We’ll behave! Double pinky promise!”
Yeah, what could possibly go wrong?
I’m starting to think that AI CEOs are actually retarded, or at least they think the rest of us are.
If you’re seriously going to govern AI, you’d need an independent third-party body made up of multiple competing groups like scientists, researchers, cybersecurity experts, and human rights advocates. And the people in this body need to have zero financial or professional conflicts of interest with frontier AI companies.
Companies like Anthropic, OpenAI, and Google can absolutely have a seat at the table; they understand the technology and should inform the standards, but they don’t get to control them.
Because if I can poke holes in the CEO of Anthropic’s plan in seven minutes, what are we doing?
You don’t get to create the problem, profit from it, and have a controlling say in the solution.
If you liked this blog post, subscribe to The Zach Files on Substack to get these posts sent straight to your inbox.