The Suicide Region: Option Games and the Race to Artificial General Intelligence
Abstract
Standard real options theory predicts delay in exercising the option to invest or deploy when extreme asset volatility or technological uncertainty are present. However, in the current race to develop artificial general intelligence (AGI), sovereign actors are exhibiting behaviors contrary to theoretical predictions: the US and China are accelerating AI investment despite acknowledging the potential for catastrophic failure from AGI misalignment. We resolve this puzzle by formalizing the AGI race as a continuous-time preemption game with endogenous existential risk. In our model, the cost of failure is no longer bounded only by the sunk cost of investment (I), but rather a systemic ruin parameter (D) that is correlated with development velocity and shared globally. As the disutility of catastrophe is embedded in both players' payoffs, the risk term mathematically cancels out of the equilibrium indifference condition. This creates a "suicide region" in the investment space where competitive pressures force rational agents to deploy AGI systems early, despite a negative risk-adjusted net present value. Furthermore, we show that "warning shots" (sub-existential disasters) will fail to deter AGI acceleration, as the winner-takes-all nature of the race remains intact. The race can only be halted if the cost of ruin is internalized, making safety research a prerequisite for economic viability. We derive the critical private liability threshold required to restore the option value of waiting and propose mechanism design interventions that can better ensure safe AGI research and socially responsible deployment.
Summary
This paper addresses the apparent paradox in the race to Artificial General Intelligence (AGI): despite the extreme uncertainty and potential for catastrophic failure, major players like the US and China are accelerating investment, contrary to what standard real options theory would predict. The authors resolve this puzzle by modeling the AGI race as a continuous-time preemption game with an endogenous existential risk parameter (D). They argue that the standard real options framework fails to account for the fact that the cost of AGI failure extends beyond sunk investment costs (I) to include potential systemic ruin (D), which is shared globally. The core finding is the existence of a "suicide region" in the investment space. In this region, competitive pressures force rational agents to deploy AGI early, even with a negative risk-adjusted net present value. This is because the disutility of catastrophe (D) is embedded in both players' payoffs and cancels out in the equilibrium indifference condition. The paper further demonstrates that "warning shots" (sub-existential disasters) are unlikely to deter AGI acceleration. The authors propose mechanism design interventions, such as privatizing the cost of ruin through liability thresholds or relaxing the winner-takes-all nature of the race, to restore the option value of waiting and promote safer AGI development. This matters to the field because it provides a formal economic framework for understanding and addressing the risks of the AGI race, offering insights into potential policy interventions.
Key Insights
- •The paper introduces the concept of a "suicide region" where the fear of preemption outweighs the economic rationality of waiting, leading to premature AGI deployment despite potential catastrophic consequences.
- •The "cancellation effect" is a key theoretical contribution: the shared cost of systemic ruin cancels out in the equilibrium indifference condition, rendering the magnitude of the disaster ineffective as a deterrent.
- •The paper demonstrates that "warning shots" or sub-existential disasters are unlikely to significantly alter the dynamics of the AGI race without fundamental changes to the payoff structure.
- •The analysis highlights the "Saviour Premium," where asymmetric beliefs about safety standards (π_self > π_rival) can further incentivize early deployment, exacerbating the risks of the AGI race. This reduces the preemption threshold: V*_p,saviour = I - D(π_self - π_rival) / π_self < V*_p = I / π.
- •The paper derives a critical private liability threshold (D_private) required to internalize the externality of AI risk and restore the option value of waiting, demonstrating that the race may be halted if a regulator can set D_private high enough so that the preemption threshold meets or exceeds the survival threshold (V*_p,liability ≥ V*_s).
- •The model reveals that the preemption threshold (V*_p) is independent of the global cost of ruin (D), indicating that the potential for catastrophic misalignment does not inherently deter players from deploying AGI.
Practical Implications
- •The research suggests that voluntary coordination mechanisms, such as pause agreements, are unlikely to be effective in the AGI race because they do not address the underlying asymmetric Leader-Follower payoffs.
- •Policymakers can leverage the derived critical liability threshold (D_private ≥ S * (I + (1 - π) * D_social) / (1 - S)) to design regulatory interventions, such as catastrophe bonds or strict liability torts, that internalize the risk of AGI misalignment and incentivize safer development practices.
- •The paper highlights the importance of relaxing the winner-takes-all nature of the AGI race, for example, through "windfall clauses" that redistribute the economic benefits of AGI success, thereby diminishing the fear of missing out and increasing the value of the option to wait.
- •The analysis suggests that robust verification processes, such as hardware-level verification or blockchain-based monitoring, can reduce strategic uncertainty and delay deployment during the early stages of AGI research, but may also trigger a sprint towards deployment during the endgame phase.
- •Future research should focus on developing effective mechanisms for verifying and enforcing safety standards, as well as exploring alternative payoff structures that promote cooperation and shared benefits in the AGI race.