On Nagel, "What Is It Like to Be a Bat?"; Liao, "The Moral Status and Rights of Artificial Intelligence"; Risse, "Structural Stalemate."
The question. If we cannot currently determine whether sophisticated AI systems are conscious or have morally relevant properties, what governance obligations does that uncertainty create — and who has the standing to adjudicate such questions institutionally?
The limits of our own conceptions
If we cannot tell whether a system is conscious because we have no first-person access to its mind, we cannot simply apply human-centric moral and legal standards to it. Being human, we risk projecting our own cognitive structures onto something with an entirely different substrate.
This is Nagel's point. His realism about the subjective domain implies a belief in facts beyond the reach of human concepts — and the subjective experience, and possible moral status, of a non-biological system may be exactly that kind of fact. We cannot comprehend what it is like to be a highly complex parallel-processing network.
Obligations under deep uncertainty
Under two burdens at once — philosophical uncertainty about whether a system has experience, and technical uncertainty about whether it is genuinely aligned or merely behaving as though it were — the historical paradigm of treating a technology as safe until proven unsafe has to be reversed.
Developers of frontier models should carry the burden of proof: rigorous safety cases, and published if-then commitments that trigger specific protocols when red-line capabilities appear. And because progress is rapid and unpredictable, static legislation is conceptually insufficient. Regulation has to be milestone-triggered — tightening automatically when training compute or capability accelerates, loosening if it plateaus.
Liability, licensing, security
Regulators should hold frontier developers and owners strictly liable for harms arising from systems whose high-dimensional reasoning they cannot fully predict or interpret. Governments have a sovereign obligation to license frontier development and to restrict AI autonomy in critical infrastructure and military roles. And to stop a highly capable system being stolen or escaping, clusters need security robust enough to withstand state-level espionage — airgapped datacentres, hardware-level encryption, SCIF-based development.