None
TL
Should we "pace" AI self-improvement?
['Tim Fist']
Noahpinion
In our view, the letter is implicitly arguing three things:Frontier AI companies are close to fully automating AI R&D.
Because some disagreements about whether to pace AI R&D stem from different predictions about what level of AI R&D automation (and resulting acceleration) is even possible, identifying concrete thresholds might allow for different camps to reach positive-sum compromises.
Accelerating research that would make further AI R&D automation safer, either by: Improving the safety of AI models directly (e.g., via developing AI control protocols), or Boosting societal resilience to make the negative consequences of new AI capabilities less acute (e.g., via using AI to patch open-source code vulnerabilities).
For the reasons we outline above, doing this kind of reallocation could unlock the benefits of AI more broadly than if frontier AI companies focus exclusively on internal AI R&D.
The US government can prepare for this kind of targeted pacing today by:Providing transparency into automated AI R&D Improving state capacity to understand and respond to automated AI R&D Developing a risk management strategy for automated AI R&D that accelerates defensive and commercial AI uses Accelerating the development of AI verification technology Investing in AI resilience Extending the US AI lead to give the US more time to manage AI R&D automation risks Creating option value for international cooperation on managing automated AI R&D risksIn the next post, we’ll provide 23 specific policy ideas to achieve these seven goals with minimal downside even if the risks of automated AI R&D turn out to be low.