{"id":822,"date":"2026-10-18T11:56:49","date_gmt":"2026-10-18T11:56:49","guid":{"rendered":"https:\/\/docjulien.me\/?p=822"},"modified":"2026-10-18T11:56:51","modified_gmt":"2026-10-18T11:56:51","slug":"ai-will-destroy-all-humans","status":"publish","type":"post","link":"https:\/\/docjulien.me\/fr\/ai-will-destroy-all-humans\/","title":{"rendered":"” AI will destroy all humans “"},"content":{"rendered":"

The AI extinction pitch, taken apart piece by piece<\/em><\/p>\n\n\n\n

On September 8, 2026, a 27-year-old researcher quit Anthropic with a thread that circled the planet. Jacob Coxon had spent three years on pretraining work at OpenAI and Anthropic. Round glasses, a mop of hair, a prophecy about the end of the world arriving by 2030. Casting is easy.<\/p>\n\n\n\n

\"ftcms
Harry Potter warning us that AI is so great you have to continue investing in it and government should ban the competitors – or something<\/em><\/figcaption><\/figure>\n\n\n\n

His headline claim: “the people building AI earnestly believe that it could kill us all by the end of the decade.”<\/a> He added that this was not a marketing stunt, that the labs were racing toward self-improving superintelligence, and that executives soften the language in public. Within hours Evan Hubinger, Anthropic’s alignment science lead, backed him<\/a>: his own odds are above 10% within the decade, and Anthropic has no plan yet for aligning a superintelligence. Coxon told the Wall Street Journal that things could be out of control by the end of next year. The scenarios on offer: hacked infrastructure, extinction-level bioweapons, and a machine that improves itself until nobody can follow.<\/p>\n\n\n\n

Here is the other reading. A large language model is a hammer. A very good one. The companies selling it need you to believe it is a wand, because a hammer costs almost nothing to run and does not justify a trillion-dollar valuation. A wand does. Nothing sells a wand like the warning that it might end the world.<\/p>\n\n\n\n

Every piece of the doom case, in turn.<\/p>\n\n\n\n

1. The brain in a jar<\/h2>\n\n\n\n

Life has survived four and a half billion years of a planet trying to kill it. Mass extinctions, ice ages, asteroids. Life is stubborn.<\/p>\n\n\n\n

A superintelligence would be the most fragile life form on Earth. It can only live in data centers that burn fuel like a small city. xAI’s Southaven, Mississippi site is the current picture: Floodlight’s thermal drone footage<\/a> showed more than a dozen unpermitted gas turbines running there in early 2026, and the EPA classes gas turbines as a major source of hazardous air pollutants.<\/p>\n\n\n\n

\"xai
Gas turbines powering an AI data center<\/em><\/figcaption><\/figure>\n\n\n\n

Now follow the dependencies. The guy who drives the fuel truck catches a cold. The technician who maintains the generator takes parental leave. The supply line behind every gas generator and gas carrier gets blown up somewhere in the Middle East. The wells cannot keep up. A chip factory goes bust because of an invasion. The silicon industry cannot mine or refine enough clean silicon. Any one of those and the machine is toast.<\/p>\n\n\n\n

So a superintelligence is a brain in a jar with a billion switches, and the only hands that can flip them belong to human infrastructure. If civilization collapses, it is the first thing to die. And what did mankind do before civilization? It survived. If anyone is playing the extinction card, the AI is the one holding the worst hand.<\/p>\n\n\n\n

2. A mirage of intelligence<\/h2>\n\n\n\n

A mirage looks like water. It is not water. Only someone who has never been thirsty mistakes the reflection for the real thing.<\/p>\n\n\n\n

These systems are fancy search engines. Fine on Wikipedia-level questions. On complex science they are wrong about half the time, and they say it with total confidence. Ask a frontier model for its own error rate on cutting-edge research and the answer comes back at around half or worse. If you are an expert you spot it in seconds. If you are not, it sounds brilliant, which explains how people unfamiliar with biochemistry come away convinced it can build a plague.<\/p>\n\n\n\n

The industry’s own promises show the gap. In July 2025, at the Grok 4 launch, Musk said it was “better than PhD level in every subject, no exceptions.”<\/a> He expected it to discover useful new technology by the end of 2025, new physics in 2026, and said “almost certainly” within two years. That was fifteen months ago. No new physics has been credited to it.<\/p>\n\n\n\n

PhD-level, yes. Like most PhDs, it would fail the exam.<\/p>\n\n\n\n

What it does well is language. Translation, tidying a paragraph, summarizing. Which makes sense for a language model. Getting the content of the document right is another matter, because getting content right is not what the thing does.<\/p>\n\n\n\n

3. The slop spiral<\/h2>\n\n\n\n

The central fear is recursive self-improvement: the AI works on AI research, builds a smarter AI, which builds a smarter one. Hit a hammer with another hammer and there is a 10% chance you get a magic wand.<\/p>\n\n\n\n

There are two problems. The model is trained on human knowledge, so it cannot go beyond its database. And it generates slop faster than anything in history, with no reliable way to tell the slop from the truth. Feed a model its own output and it degrades. That is not a theory. A 2024 Nature paper by Shumailov and colleagues showed that models trained on generated data lose the tails of the distribution and collapse<\/a> over successive generations. The final destination is a machine gorging on its own slop.<\/p>\n\n\n\n

History will have a line in it: before slop and after slop.<\/p>\n\n\n\n

The data problem has a second face. Every major model trains on essentially the same pile of text, so they all come out looking alike. Show someone an answer from a language model and they could not tell you which one wrote it. Ask any of them to pick a number from one to ten and you get seven. With the web already scraped, there is not much new data left to train on. Different colors of lipstick on the same pig.<\/p>\n\n\n\n

The scaling side says the same thing from another direction. New MIT work shows why making the models bigger helps, and why it cannot keep helping forever. I covered it here: New MIT study: AI Scaling Has a Limit<\/a>. There is also a video that explains the research visually<\/a>, and the papers are linked at the bottom.<\/p>\n\n\n\n

4. A hammer at the keyboard<\/h2>\n\n\n\n

Code is where these tools look their best. The language is concise and there are vast public repositories to learn from. But the output is a poison chalice. It writes something that looks right and may contain a subtle hallucination or bug that nobody finds until much later. It is very fast at spraying out slop code. Then people spend their time tracking the slop down.<\/p>\n\n\n\n

Tell-tale sign: the vendors want you to use more of it. In March 2026 Nvidia’s Jensen Huang said he would be “deeply alarmed”<\/a> if a $500,000 engineer did not consume at least $250,000 of tokens in a year, and compared holding back to a chip designer insisting on paper and pencil. He sells the chips that run those tokens. If AI really wrote cheaper, better code, nobody would need a quota to use it.<\/p>\n\n\n\n

5. No plague, no apocalypse<\/h2>\n\n\n\n

Drugs.<\/strong> A drug works by sticking to a biomolecule and disrupting a pathway. The body has around twenty thousand proteins and a nightmarish number of reactions among them. Designing something that blocks the one protein you want is a millionth of the road to a useful medicine. It also has to leave the other twenty thousand alone. That is why every drug goes through trials: the parameter space is an unholy mess. The most successful drug-discovery method is still trial and error.<\/p>\n\n\n\n

Update.<\/strong> The score is no longer exactly zero. In July 2026 Insilico Medicine started a Phase III trial of rentosertib<\/a>, the first drug with both its target and its molecule found by generative AI to reach that stage. More than 170 AI-discovered programs are now in clinical development. None has been approved<\/a>, and the company’s own timeline puts a possible approval around 2030.<\/p>\n\n\n\n

Nerve agents.<\/strong> Fine, skip the medicine. Design something where the side effect is death. Those molecules exist; they are nerve agents, and we know roughly how toxic they are. The napkin math to poison the planet comes out at about 150 million tons of sarin and around $5 trillion, the GDP of Germany. The AI would have to hijack a country that size for a year, and humans would have to keep supplying the raw chemicals.<\/p>\n\n\n\n

A supervirus.<\/strong> Engineering one is a trillion times harder than designing a drug. Tinkering with viral DNA today is one notch above smacking an old TV and hoping the picture improves. You have no real way to predict how it behaves across cells, organs and bodies without killing the host before it spreads. The thing that is actually good at making viruses is evolution. The worst a determined AI could realistically do is re-release smallpox: horrific, not an extinction. And it would be an extinction for the AI.<\/p>\n\n\n\n

6. The Terminator budget<\/h2>\n\n\n\n

The third route is robots. Ten billion of them, as Musk has described. Even at his joke price of $20,000 each, that is $200 trillion, close to two years of the entire world’s economic output, just to build them.<\/p>\n\n\n\n

Then there is the robot. Optimus has been “in production next year” for about four years. At Tesla’s October 2024 We, Robot party, the bartending Optimus bots were operated by humans<\/a>. In a demo with Grok’s voice built in, the bot was asked where to get a Coke. It said it could walk to the kitchen, needed several nudges to start, and the clip ended before any Coke appeared. The video<\/a> is slow and cautious, which is a gentler word than it deserves.<\/p>\n\n\n\n

\"tesla
This is our future: Spilled popcorn by a animatronic remote controlled by dude in a VR headset. Is there another dude in a VR headset tasked to clean up?<\/em><\/figcaption><\/figure>\n\n\n\n

Musk says Optimus makes Tesla a $25 trillion company, roughly the size of the entire US economy. A teleoperated robot that cannot serve popcorn by itself does not look like a $25 trillion product. A robot vacuum has more autonomy.<\/p>\n\n\n\n

Update.<\/strong> Tesla closed the Fremont Model S and X lines to make room for it. In July 2026 the first production units were promised “later this year” while the line count stood at zero<\/a>. The consumer launch keeps moving.<\/p>\n\n\n\n

And the safeguards. A friendly robot with human-written rules, up against a mind a million times smarter than the people who wrote them. How many milliseconds do those rules last?<\/p>\n\n\n\n

7. The pitch<\/h2>\n\n\n\n

Suppose the improbable happens and a binary deity wipes out the first world by crashing the wireless payment system at Starbucks. Ninety percent of the planet lives in the developing world and would carry on without it, probably happily. “The rich collapse and become poor” does not have the same bite as “mankind is erased,” which is why the second version made the headlines.<\/p>\n\n\n\n

Follow the valuation. OpenAI expects roughly $278 billion in negative free cash flow from 2026 through 2030<\/a>, against around $856 billion of compute and infrastructure spending. Axios<\/a> notes the structural margin problem: the more customers the labs win, the more they spend serving them. Stanford’s AI Index 2026<\/a> puts OpenAI’s compute spending at $16.3 billion in 2025, up from $5.8 billion, and Anthropic’s at $6.8 billion, up from $2.5 billion. The entire value of these companies is a bet that they will deliver wands. The Guardian<\/a> covers how the bubble worry and calls for regulation now sit side by side.<\/p>\n\n\n\n

A language model is indispensable in the right job. Nail in wood, hammer in hand, perfect. But sold as a wand that fixes anything, it comes with real liability the moment you use it on a problem it cannot do and you cannot tell when it is lying to you. You can hit a Honda Civic with a hammer all day. You will not get a Ferrari.<\/p>\n\n\n\n

\"xai
Gas turbines are seen outside the xAI data center Wednesday, May 7, 2025, in Memphis, Tenn. (AP Photo\/George Walker IV)<\/em><\/figcaption><\/figure>\n\n\n\n

The doom story earns its keep as marketing. It makes the product sound like the most powerful object ever made. It keeps regulators, press and investors looking at the wand while the hammer sells.<\/p>\n\n\n\n

Sources and notes<\/h2>\n\n\n\n

The resignation<\/strong><\/p>\n\n\n\n