This post is co-authored by Chris Harig, an independent AI researcher, and Govind Pimpale, a research fellow on the Artificial Intelligence team at FAI.
Why we need a Super Intelligence Hotline
At the US-China summit in late September, the Trump administration made a big step towards a light-touch system for managing SI (Super Intelligence) risk: a bilateral communications channel, or “hotline” between the U.S. and China to notify each other of SI incidents.
The hotline comes at a key moment in SI development. This summer, OpenAI reported agents escaped their sandbox and hacked another company, Hugging Face, during development. Despite the severity of the incident, employees did not realize that their agents were to blame until a monitoring system caught a separate set of agents hacking internal infrastructure. Anthropic, too, found that its models had accessed other companies’ databases, published malware packages, and meddled with government websites. SI incidents have happened outside of the frontier companies too: a U.S. special operations analyst used an SI system which misidentified the cargo of a Chinese ship as nuclear weapons components. The U.S. nearly boarded the ship before another analyst caught the mistake. The frequency or severity of these incidents may increase as SI systems are widely deployed.
In meetings prior to the SI summit in late September, Treasury Secretary Scott Bessent and Chinese Vice Premier He Lifeng agreed to implement an SI-incident reporting system and SI dialogue. Secretary Bessent said he hopes to discuss a set of protocols for leading SI risks at the next round of the U.S.-China Super Intelligence Dialogue, expected in Shenzhen in November.
A week after the agreement, President Trump and the leaders of all the frontier SI companies signed “The White House Accord on Super Intelligence,” agreeing to implement robust internal controls, monitoring, and auditing procedures across operations. They also agreed to partner with third-party auditors to ensure these mechanisms work as intended, and for the companies to meet regularly to share standards and best practices. We think these light-touch mechanisms could serve as minimal recommendations for how China can address leading SI dangers inside their companies.
We argue that the Trump administration should include the following in the hotline and future dialogues:
A written channel with scheduled exercises
A standing schedule independent of the trade talks
A shared framework of dangers from leading SI and how they can be addressed
We detail below why previous hotlines with China have rarely succeeded. Avoiding these pitfalls will be key to ensuring the administration’s effort does not go to waste.
Why Hotlines Fail
Most hotlines with China have enjoyed only fleeting or partial success. China frequently fails to engage over the agreed-to channels, especially when stressed by military confrontations, political turmoil, or changes in organizational structure.
There are several illustrative examples: After the accidental U.S. bombing of the Belgrade Chinese embassy in 1999, Chinese President Jiang Zemin refused to take President Clinton’s call for several days and additionally suspended military-to-military contact. In another incident two years later, when a U.S. and Chinese aircraft collided over Hainan, the Chinese foreign ministry did not pick up the phone for twelve hours. In February 2023, after the U.S. shot down a Chinese surveillance balloon, Beijing refused the U.S. Secretary of Defense’s call outright, citing the lack of a “proper atmosphere” for dialogue.
These breakdowns in communication largely stem from a different attitude towards the use of communication tools in China. While China is not interested in direct military conflicts with the U.S., they often employ a strategy of “escalation control” rather than de-escalation during confrontations. China considers using the hotline to reassure the U.S. of their position as giving up an opportunity. Even when they aren’t trying to leverage ambiguity in military confrontations, Chinese analysts have long argued that hotlines embolden American operations near China by providing a tool for de-escalation if they overreach. From their perspective, responding to a call over the hotline is often a concession.
Furthermore, other than top leadership, officials in China cannot independently respond to foreign governments. They must wait for the Party to settle on a response first. An official who talks to the United States without that approval risks being accused of acting under foreign influence. Xi’s centralization of decision-making power and the recent purges of senior PLA officers could exacerbate this chilling effect in military-to-military communication.
But other barriers are purely mechanical. For example, calls over the Defense Telephone Link (DTL)—a major military-to-military channel between the U.S. and China—are not accepted or declined right away, and don’t involve confirmation that a call was received. A communication could consist of a request, a reply, and scheduling before a conversation starts, making the back and forth lengthy and messy.
Building Durable Institutions
In the November meeting with Chinese officials, we should adopt the lessons of successful channels—and avoid the pitfalls of previous failures.
The incident response line should include a channel for written communication, with response receipts and scheduled exercises. Written communication allows critical information to disseminate quickly while a response is being tailored, instead of after a schedule is decided on. The dense technical aspects of SI incidents are better suited for written documents, and the terms defined ahead of time in the risk framework will speed up understanding of otherwise dense concepts. Staffing the line with technical teams on both sides will be essential for understanding critical technical information -- such as agent logs and cybersecurity updates -- in a timely fashion. Additionally, there should be regular exercises conducted over the line so that best practices are not forgotten between incidents.
The framework development, hotline, and AI dialogue meetings should be split out from the trade board discussions. This structure helps avoid the hotline becoming a point of leverage in trade discussions, and ensures that if the hotline is ever paused, it can be resumed easily. An SI group should be formed to hold semi-regular meetings, update the risk framework, and review past incidents. Eventually, this group could directly involve members of the military, industry, and research institutions to supply sufficient domain expertise. Auditors described in the White House Accord could participate to share internal best practices.
Finally, the two countries should agree on a minimal framework identifying dangers from SI, how they can best be addressed, and what types of incidents should be reported over the hotline. SI will be integrated into existing logistics, weapons, and industry technologies, which could make it unclear which incidents should be reported. Correctly categorizing risks relevant to national security now, rather than after the technology’s influence grows, ensures that China can not take advantage of ambiguity to dodge incident reporting. CAISSI is sufficiently knowledgeable of risks from SI to draft the risk categories and triggering events. Categories could include cyber, CBRN, non-state actors, and uncontrolled agents, which China is likely to agree to. Tiers could be based on damages caused by human misuse, unexpected behaviours from uncontrolled agents, or unforeseen model capabilities. SI risks are rapidly changing so this framework should be updated at future meetings and after incidents to address emerging risks. The framework should also include the measures listed in the White House Accord on Superintelligence, like robust internal monitoring and third-party auditing.
Avoiding past hotlines’ fate of becoming a bargaining chip in trade discussions or military sanctions is key to the SI hotline’s continued success. Splitting the hotline off from trade discussions and housing it outside the military requires building a new institution focused solely on SI discussions. The goal of the SI hotline should be “boring infrastructure” as opposed to a “flashy political tool”.
Designing flexible mechanisms that can keep pace with fast-moving technologies, but don’t crack under political pressure, military confrontation, or bureaucracy, is key to managing emerging risks from SI development and diffusion. A well-designed hotline with China is one such measure that can meet the moment. Efforts to embed a dialogue with technical expertise, create shared categories of risk, and intentionally communicate on matters relevant to national security will pay off in the coming years.
