Annex 1
Proposed institutional structures
IASC Organizational Structure:
Council: Each signatory to the treaty can appoint a representative member to the Council. Each member has equal voting rights.
Executive Board: Analogous to the UN Security Council, this consists of representatives of major member states and supranational organizations, which would all be permanent members with vetoes on decisions taken by the Executive Board. The Executive Board also includes non-permanent representatives elected by a two-thirds majority of the Council.
General Secretary: Oversees the running of IASC and is appointed by a supermajority (75%) vote of the Executive Board. The General Secretary sits for a five-year term and can have a maximum of two terms in office. The General Secretary must have multiple duties and powers, including but not limited to:
Lowering the compute thresholds: The General Secretary formally makes the decision to lower the compute limits established in the Multi-Threshold System, on the advice of the Advisory Committee, following a report of the AI Scientific Measurement Team;
Revocation of the registration status of an AI organization or company: The General Secretary formally makes this decision on the advice of the Advisory Committee, following a report of the Global Oversight Team;
Revocation of the registration status of companies with a particular national authority: The General Secretary formally makes this decision on the advice of the Advisory Committee, following a report of the Global Oversight Team;
Ordering the removal of a senior officer of the GUARD lab: The General Secretary formally makes this decision on the advice of the Advisory Committee, following a report of the Global Oversight Team;
Approval of specific limited exemptions to the Medium Compute Limit, established in the Multi-Threshold System: The General Secretary formally makes this decision on the advice of the Advisory Committee, following a report of the AI Risk Analysis Team. Such exemptions could only be granted to licensed organizations for specific narrow model types, under strict safety and ethical conditions and subject to regular review;
Recommending/setting the annual budget for the GUARD lab: The General Secretary formally makes this decision on the advice of the Advisory Committee, following a report of the Global Oversight Team.
Advisory Committee: A limited group of AI scientists that have been appointed by the Council. The Committee provides recommendations on major decisions, based on reports produced by the teams reporting to the General Secretary.
AI Global Oversight Team: A directorate within IASC that:
Oversees GUARD, including its budget, hiring, strategic plan, operations, and provides reports on this to the General Secretary;
Audits and assists national regulators with implementing new guidance from IASC;
Maintains a list of licensed AI models approved by national regulators within the internationally set middle compute threshold;
Undertakes international investigations into undeclared development of major AI models.
AI Scientific Measurement Team: A directorate within IASC that:
Maintains international measures/standards for AI capabilities and risks;
Provides reports on progress in the development of AI science and safety, including on boundedness, corrigibility, and alignment.
AI Risk Analysis Team: A directorate within IASC that:
Provides advice on the overall level of existential and catastrophic risk as a result of AI, through specialized investigations and assessments.
GUARD Organizational Structure:
Managing Director: Appointed by IASC Council for a 10-year term, and is responsible for overseeing the research, operations, and strategic direction of GUARD.
Executive Board: Appointed by IASC Council for a 5-year term, and is responsible for overseeing the Managing Director and the strategic direction of GUARD. Each board member has a single equal vote on issues and a 75% majority is required for all decisions.
Scientific Committee: Appointed by IASC Council for a 10-year term and is responsible for providing specialized advice on AI research and development to the Executive Board and Managing Director.
Financial Committee: Composed of representatives from the national administrations of treaty signatories and is responsible for providing advice on all issues relating to financial contributions and the lab’s budget and expenditure.
Risk Management Committee: Runs internal risk management function; collaborates with IASC audit functions,
Operational and Research Teams: A combination of various directorates within GUARD that are responsible for delivering on its strategy, to include divisions such as: (1) Directorate of Alignment; (2) Directorate of Boundedness; (3) Directorate of Capabilities Assessment and Development; (4) Directorate of Fundamental AI Research; (5) Internal Safety Audit Directorate; (6) Finance Directorate; (7) Information and Technical Security Team.
IAT Organizational Structure:
Chairperson: Chaired by a representative selected by the Council on a rolling 10-year term. The Chairperson facilitates meetings, guides dispute resolutions, and represents the IAT externally. The Chairperson can also unilaterally refer cases to the Rapid Response Panel.
The Court: Consists of 31 judges appointed by IASC Council to serve six-year renewable terms, with two main elements:
Chambers System: Modeled on the European Court of Justice, by default cases are heard by a Chamber of 5 randomly selected judges or in significant cases (as defined through treaty terms) by a Grand Chamber of 15 judges.
Advocates General Procedure: To aid in the processing of cases, Advocates General are appointed by the General Secretary to provide independent opinions on the legal issues in cases before the court on all issues. If the Advocate General makes a finding that there are no substantive new issues of law in the case, they shall refer to any advice and decisions that had been made on any previous relevant cases.
Risk Assessment Panel: A panel of 2 judges and 3 experts drawn from IASC’s AI Risk Analysis Team with the responsibility of:
Being the first point of contact between a submitted case and the IAT;
Making a rapid assessment about the risks of the case being subject to a prolonged arbitration process, and to make a decision on whether to refer the case to the Rapid Response Panel;
Setting the time limit of a Rapid Response Panel determination if needed.
Rapid Response Panel: A specialist panel capable of convening swiftly to address urgent cases, formed of 3 judges. By default, Rapid Response Panels have a maximum of 30 days to take preliminary action (e.g., a temporary restraining order). If initial action is not made within the allotted period of time, then the case is referred to the Risk Assessment Panel to make a snap judgment on.
Appellate Body: Consists of 7 members serving staggered four-year terms, appointed by IASC Council.
All judgments made by the IAT are legally binding within the framework of international law that the AI Treaty establishes, however, findings of a Court or Rapid Response Panel may be appealed. The Appellate Body can uphold, modify, or reverse legal findings and conclusions.
Annex 2
Reasoning underpinning the Multi-Threshold System
The Upper Limit is set approximately at the highest amount of compute that any AI model has been trained to date. Until significant progress has been made on safety research, AI capabilities should not be further advanced, hence nobody is permitted to train models above the Upper Limit.
It’s not possible to be sure that systems at current capabilities levels are safe. In this proposal the most powerful systems are trained in the GUARD lab, providing access to the APIs of models that are reliably safe, and hence only the GUARD lab can train models above the Medium Limit.
The Lower Limit is placed at a level where development of dangerous AI systems seems plausibly possible. Above this limit developers are required to obtain licensing.
The maximum permitted performance of computing clusters are calculated keeping the following aims in mind:
We want to ensure that no actor apart from GUARD can quickly get in range of the Upper Compute Limit, for example by running an illegal training run, surpassing the training compute limits with relatively low timeframe for detection.
We want to ensure that unlicensed actors can not quickly get in range of the Medium Limit, in which only licensed actors and GUARD is permitted to train models, since these will have a high level of capabilities, and without proper safety best practices may be dangerous.
We do want licensed actors to be able to train models permitted for them within reasonable timeframes.
We do want unlicensed actors to be able to train models permitted for them within reasonable timeframes.
We don’t want to ban commonly-owned personal computing devices.
To achieve these aims, we can focus on the amount of time it takes to train a particular illustrative model given both the total desired model size and the compute capabilities of a computing cluster at a lower limit.
In this system, we have a difference in order of magnitude of 8 between the Upper Limit Compute limit and the Medium Limit Computing Cluster limit, meaning that it would take a licensed cluster 3.2 years to breach the Upper Limit in an illegal training run, giving authorities ample time to intervene. However, licensed actors could still train any permitted model within 12 days, since there is a difference in magnitude of 6 between the Medium Limit’s Compute and Cluster limits.
We also have a difference in order of magnitude of 8 between the Medium Limit Compute limit and the Medium Limit Compute Limit and the Lower Limit Computing Cluster limit, meaning that unlicensed actors would take 3.2 years to breach the Medium Limit (and theoretically hundreds of years to breach the Upper Limit). However, unlicensed actors could still train any permitted model within 12 days, since there is a difference in magnitude of 6 between the Lower Limit’s Compute and Cluster limit.
Annex 3
Some interventions we considered but decided against
Regulating Model Size
Model size, as measured in the number of parameters that an AI model has, is a predictor of model performance and capabilities. But we found that compute is a preferable proxy for regulation for two reasons: i) model size strongly correlates with training compute, due to scaling laws, meaning that model size is not a more efficient proxy for capabilities than training compute is; ii) hardware is easier to monitor, and since few companies can afford the huge computational resources necessary to train frontier models, regulating compute means only having to monitor these few actors.
Leaving Advanced AI Development Decentralised
While we do advocate for a licensed development of frontier models by private companies, the risks from allowing a competition - whether it be between companies or nation states - to develop the most advanced AI models are simply too high to be tolerated. Proliferation of advanced models would mean a proliferation of opportunities for serious loss-of-control or weaponization to take place.
Regulating Training Data Breadth
Regulating training datasets is appealing since how varied a model’s training dataset is may predict how varied the model’s capabilities are, and also because volume of training data also predicts a model’s performance. We considered multiple options of AI training data regulation to achieve different objectives, and will share them in future iterations of this project.
A ‘Formula One’ Style Regulatory Regime
One potential criteria for awarding licenses to frontier developers would be to make their licenses contingent upon a track record of responsible and safe development. With regards to frontier AI development, this would have the advantage of making it difficult for younger AI companies that don’t yet have a track record of frontier development to join the licensing system. This would limit the number of companies that can join the frontier race, thereby decreasing the chance of catastrophe from race dynamics.
However, this system makes a dangerous and unjustified assumption: that past track record is a strong predictor of future safety practices; unfortunately, this claim is not justifiable at this stage of maturity of the AI industry. In addition, such a system increases the prospect of regulatory capture on behalf of the frontier labs already competing.
