GCR Policy Newsletter (20 March 2023)
Governing existential risk in the UN system and upgrading national risk assessments for GCR
This twice-monthly newsletter highlights the latest research and news on global catastrophic risk. It looks at policy efforts around the world to reduce the risk and policy-relevant research from the field of studies.
GCR in the media
“Thiel is simply wrong if he thinks that ‘slow it down’ is the only response…But it’s also clear that the old Facebook motto of ‘move fast and break things’ won’t work. Just speeding up technology won’t be enough to keep us safe. We need sensible domestic regulation: supporting the green transition, raising safety requirements in biological labs, and ensuring that high-risk AI systems go through safety tests.” There’s no libertarian approach to preventing the end of the world (Vox)
“As horrible as Covid has been — it remains one of the leading causes of death in the United States — it is not the worst-case scenario. There are viruses with case fatality rates twice, 10 times or even greater than that of Covid, such as H5N1 influenza (bird flu), Nipah and Ebola. Fortunately, those viruses have not developed the capacity for efficient human-to-human respiratory spread. A concern is whether a new viral strain with higher case fatality will also develop the capacity for rapid spread among people. There is growing global concern over H5N1’s spread in animals — a development that governments must track and prepare for, and which all the more should broadly spur vigorous new pandemic preparedness efforts. The world needs to be prepared for the next Disease X, something capable of causing global catastrophic risks.” How to Prepare for the Next Pandemic (New York Times)
“We need to get serious about biosecurity — and quickly. Biological risks have evolved dramatically in a short time, and governments need to act, both at home and together. Advances in biotechnology have potentially made it easier to create or modify deadly pathogens, lowering the barriers for adversarial states and extremist organisations to develop biological weapons…With a new cabinet facing so many challenges, ruthless prioritisation is needed to safeguard the UK’s economic wellbeing and national security. Biosecurity needs to be up there — delivered through an ambitious strategy and dogged implementation of its recommendations.” The UK must prepare for the biosecurity threats to come (Financial Times)
“Because of power and politics, most international treaties fail to meet their stated goals (as found by a recent synthesis of more than 200 studies). Flawed governance is an acceptable and manageable risk for some policy areas…But soft governance is simply not acceptable for high-risk technologies like solar radiation management initiatives. Stratospheric aerosol injection with deeply flawed governance would be like hoisting a climatic Sword of Damocles — it may allay climate risks in the short term, but when disaster strikes (and over several decades, one will) it would be catastrophic.” Dead before arrival: The governance of stratospheric aerosol injection (The Bulletin)
“There is a broad spectrum of threats to civilization, and indeed to human existence itself. Over the past decade, threats have changed. The COVID-19 pandemic also gave us a real-time visualization of how we might respond to a greater threat, for better or worse. It is a sad reality that the greatest threat to our survival, right now, comes from ourselves.” How will the world end, naturally or by our own hands? (Big Think)
“The AI-existential-threat discussions are unmoored from evolutionary biology, cognitive psychology, real AI, sociology, the history of technology and other sources of knowledge outside the theater of the imagination.” Pinker on Alignment and Intelligence as a "Magical Potion" (Richard Hanania's Newsletter)
Latest policy-relevant research
Governing existential risk in the UN system
Existing governance structures are not fit-for-purpose to address existential risk in general nor from technological development, according to a report by the Simon Institute for Longterm Governance commissioned by the United Nations Office for Disaster Risk Reduction (UNDRR). Institutional incentives lead policymakers to focus on short-term, higher-probability events that are easier to predict and more definitively linked to their national territory. Individual countries tend to take limited ownership of global risks as governance structures tend to be slow moving, reactive rather than proactive, and have been set up with mandates that focus on specific kinds of risks, but neglect existential risk, more broadly. To make progress, it will be necessary to address four priorities for reform. First, UNDRR should foster a concrete and common understanding of existential risk. Second, UNDRR, other UN agencies and member-states should strengthen existential risk governance. Third, the UN system and national governments should dedicate more resources to existential risk reduction. Fourth, UN agencies should foster fast response mechanisms to stop shock cascades. (March 2023)
Policy comment: The governance of existential risk, particularly those arising from technological development, requires the interplay between international and national levels. The international level, such as the UN system, cannot govern existential risk without the buy-in and engagement of national governments. But it can create new mechanisms or re-orient existing ones to shape norms, develop frameworks, coordinate regional and country programs, foster cooperation between nations, and convene private, public and non-government sectors. For example, the UNDRR has well-established frameworks and processes for disaster preparedness, resilience and response - but a greater conception of its role beyond natural disasters could see it take more proactive efforts on existential and global catastrophic risk.
Upgrading national risk assessments for GCR
National risk assessments (NRAs) suffer from a lack of justification and transparency around important foundational assumptions of the process and omission of almost all the largest scale risks, according a new paper by catastrophic risk researchers Matt Boyd and Nick Wilson. NRA process assumptions around time horizon, discount rate, scenario choice, and decision rule impact on risk characterization and, therefore, any subsequent ranking. A neglected set of large-scale risks are seldom included in NRAs, namely global catastrophic risks and existential threats to humanity. Under a highly conservative approach that considers only simple probability and impact metrics, the use of significant discount rates, and harms only to those currently alive at the time, these risks have likely salience far greater than their omission from national risk registers might suggest. (12 March 2023)
Policy comment: National risk assessments require various upgrades to identify and prioritize global catastrophic risks. This paper suggests a deliberative public tool that can support informed two-way communication between stakeholders and governments so that extreme risks can be better covered. In the previous newsletter, we suggested quick fixes such as using longer time horizons and factoring in uncertainty. But NRAs are just one possible mechanism for assessing GCR. And they might be not politically or bureaucratically amenable to being upgraded. Policy advocates could consider pushing for governments to develop a specific assessment of existential and global catastrophic risk outside the NRA process, including assessing the full spectrum of threats, pathways and scenarios, potentially by the national security or emergency management apparatuses.