Machine Intelligence Research Institute (MIRI)

Article I prohibits signatories from developing or deploying artificial superintelligence and from permitting others within their jurisdiction to work towards that end.

Like the nuclear nonproliferation agreements before it, the draft Agreement aims to prevent the proliferation of technologies that would be catastrophic for humanity if allowed to proceed unchecked.

The draft Agreement thus obliges states to consolidate and monitor sufficiently large clusters of AI chips (Article V), to monitor the supply chain that produces them (Article VI), and to facilitate verification of how they are used (Article VII).

We focus first on the large-scale training runs that currently produce frontier AI, setting thresholds beyond which training is monitored or prohibited.

They conclude that the only trajectory without unacceptably high levels of catastrophic harm is a scenario in which humanity develops the ability to monitor and restrict AI development (an “off switch”) and then likely uses it to put a long-term halt...