Upon reading this article by Eliezer Yudkowsky about how we can only die with dignity in the face of an AI apocalypse, I realized something rather important when discussing any potential catastrophe: what, exactly, is the mechanism of this artificial intelligence genocide?

The world faces real and dangerous problems.  But in other paradigms, the mechanism of harm is well-understood.  Nuclear weapons can “destroy the world” because they make really, really big explosions that poison the environment.  Climate change can “destroy the world” because of rising temperatures and oceans will make the planet less clement for human life.

Even in more sci-fi scenarios, like proposed nanotechnology apocalypses, there is a mechanism of destruction: uncontrollable replication.

But with artificial intelligence?  I haven’t heard anyone discuss how AIs will accomplish widespread destruction.   And as weird as this sounds, it’s probably what we should be discussing.  If we’re worried about AI misalignment causing great harm, discussing how it will happen is critical.  Granting the odds of us fixing the alignment problem are hovering around zero percent, but that doesn’t mean that an AI will be able to kill us.  They are, after all, computers, not gods.  They will not magically gain the ability to fire nuclear missiles or build killer robot factories and supply those factories with the materials needed to hurt people.  The mechanisms of destruction are essential in this scenario!

It’s a weird little situation, AI risk analysis.  Even the best people seem to be missing quite a lot.