The two routes to catastrophe this site covers most closely, nuclear war and advanced AI, meet in nuclear command and control. The good news is that there is a rare point of agreement between the great powers. The less good news is that it covers the narrowest part of the problem.
What has been agreed
In November 2024 the US and Chinese presidents agreed on the need to maintain human control over decisions to use nuclear weapons. The commitment carried into the second Trump administration: at this year's review conference of the Non-Proliferation Treaty, the US, Britain and France stated that the ultimate decision to employ nuclear weapons should never be delegated to AI.1 A senior US defence official said in September that "it remains the Department's policy that there is a human in the loop on all decisions on whether to employ nuclear weapons", with "no policy under consideration to put this decision in the hands of AI".2 China's disarmament ambassador, Shen Jian, told a UN meeting in June that weapons should remain under human control and that military AI could raise the risk of miscalculation, while arguing that safety concerns should not justify technology restrictions.2
The gap
"Human in the loop" means a person makes the final call. It does not cover what feeds that call: early-warning analysis, intelligence fusion, targeting options and the time available to choose. SIPRI notes that the US and China define human control differently, and that differing transparency about their command systems makes it hard to know at which stages of a decision human control actually applies.1
Paul Slovic and Herbert Lin, writing in the Bulletin of the Atomic Scientists, argue that current AI does not need superintelligence or malign intent to make nuclear war more likely. It can amplify human cognitive blind spots, speed up escalation, and "flood decision makers with information—some true, some false—that they struggle to evaluate but feel compelled to act on."3 A leader with minutes to decide, presented with confident machine-generated assessments, is formally in control and practically constrained.
Proposals on the table
In September a group of US and Chinese security experts, convened by Brookings with Tsinghua and Fudan universities, proposed going further: that only humans decide on cyberattacks against nuclear command systems or critical infrastructure, that the two countries set up a military AI hotline, and that they agree a shared definition of meaningful human control. Neither government has endorsed the proposals.2
A note of caution
The debate about AI risk has its sceptics, and they make a point that applies here. Sarah Kreps argues that, unlike nuclear weapons, whose effects can be modelled from yield and population, AI risk lacks established causal relationships, so nobody can yet put a credible number on it.4 The nuclear–AI link is one of the few places where the causal chain can be drawn concretely: compressed decision time, degraded or manipulated warning, and a human who must act. That is why it deserves attention now, ahead of more speculative scenarios.