GCR Policy Newsletter (1 May 2023)
A new nuclear paradigm and global bioweapons architecture
This twice-monthly newsletter highlights the latest research and news on global catastrophic risk. It looks at policy efforts around the world to reduce the risk and policy-relevant research from the field of studies
GCR in the media
“It seems extraordinarily difficult to navigate high-stakes trade-offs like these in a principled way. Policymakers don’t know which experts to turn to to understand the stakes of AI development, and there’s no scientific consensus to guide them. One of my biggest takeaways here is that we need to know more. It’s impossible to make good decisions without a clearer grasp of what we’re building, why we’re building it, what might go wrong, and how wrong it could possibly go.” What could actually kill us all? (Vox)
“We are quickly approaching a brave new world by creating a novel intelligence that we can neither predict nor understand. Researchers are currently unaware of any hard limit on AI intelligence beyond the fundamental physical limits on computation. But one thing is clear: no one party can control the behavior of an emergent AI megasystem. Global cooperation is required. A failure to investigate the problem and put effective guardrails in place could have catastrophic consequences for all of us.” Artificial Intelligence Needs Guardrails and Global Cooperation (The Wall Street Journal)
Latest policy-relevant research
Building a new nuclear paradigm
Since the Cold War ended, the strategy and practice of nuclear deterrence has grown riskier, more urgent, more dangerous, and less stable, according to Joan Rohlfing, President of the Nuclear Threat Initiative. Nuclear deterrence is an insufficient strategy for preventing the use of nuclear weapons. Instead of a considered, deliberate decision to use nuclear weapons, a mistake, misperception, accident, cyber exploit, technical failure, or false warning are equally likely pathways to nuclear use. Deterrence was never designed to address any of these threat vectors. It is time to rethink how we manage nuclear risks. A new nuclear security system must be built on the design principle that the consequences of system failure cannot threaten to end or fundamentally disrupt civilization. (25 April 2023)
Policy comment: Nuclear weapons policy, at the global or national levels, requires a paradigm shift. World capitals do not fully appreciate or understand how nuclear dynamics will be fundamentally reshaped by the development of strategic non-nuclear weaponry, the integration of disruptive and emerging technologies in military systems, and nuclear tripolarity due to the rise of China. In light of these emerging factors, defense departments of national governments should conduct a full assessment of nuclear risk, including the potential failures of nuclear deterrence and stability and how their actions are exacerbating the risk. Given the variety and complexity of the challenges, an incremental change in nuclear policy is unlikely to be sufficient. A major research initiative that brings together experts from across various relevant disciplines - such as nuclear doctrine, conventional weapons, cyber, AI, Russia, China, geopolitics - would be an important input into government thinking, particularly the changing nature of the risk and a model for a new nuclear paradigm.
Upgrading global bioweapons architecture
Human-caused biological events involving the accidental or deliberate misuse of an engineered pathogen are more likely to lead to a global catastrophic biological risk (GCBR) event than a naturally emerging pandemic, according to researchers from the Nuclear Threat Initiative. Scientists have the capacity to deliberately or inadvertently engineer pathogens that are more virulent and transmissible than what nature creates by chance, and the upper limit of damage that could be caused by a human-engineered biological event is unknown. Prevention, early detection, and rapid response are all crucial for guarding against GCBR-scale events. New approaches to disincentivize states from developing or using bioweapons have the potential to be highly effective and are crucial for strengthening global biosecurity and preventing GCBRs. (26 April 2023)
Policy comment: The current global architecture for reducing the risk of state-based biological weapon development and use is severely lacking. The Biological Weapons Convention (BWC), and the biosecurity mechanisms outside it, do not have the necessary verification, attribution and accountability measures to disincentivise states’ use of these weapons. The gaps in the architecture are well-known by member states, and a working group has been established to strengthen the BWC (after its kick-off meeting in March, it will meet twice more in 2023). Researchers and advocates, many of which have articulated various ways to bolster the BWC’s regime and protocols, should be developing specific and concrete proposals for the working group to consider and focus their political engagement in countries that will be central to its effort. For example, Brazil, France and Georgia are important because their representatives are chairing the working group, while the US is important because its delegation initially proposed establishing the working group and its engagement will drive other states’ behavior.