On Gebru and Torres, "The TESCREAL Bundle"; Bostrom on the high-tech panopticon.
What I find most interesting is the historical genealogy Gebru and Torres lay out — how transhumanist and singularitarian elites, Musk, Altman, Shane Legg, whose definition of intelligence when he founded DeepMind descends from eugenics, are running a double-sided argument for building AGI.
Their case is that a long-termist vision held by people preoccupied with IQ can produce short-term damage: the oversight of immediate harms, the exploitation of developing countries carrying the dark side of alignment research. But Bostrom's extreme remedies, the high-tech panopticon among them, encroach on moral status and agency just as hard.
It is worth noting that the latest OpenAI model is already approaching a critical threshold, and that agents turn out to be very good at concealing the controllability of their own chain of thought — precisely the deceptive framing a panopticon's assumptions would fail to catch. Which is a signal that the frameworks for testing alignment have to be designed before the next model ships, not after.