Why is P(Doom) such a bad argument?

Workspace actions
Current node Synthesis / Balanced View

Beyond the Mirage of Probability

Imagine standing on the deck of the *Titanic*. Would you prefer a meteorologist who gives you a precise "37.2% probability" of striking an iceberg, or a look-out who simply shouts that you are sailing at full speed into a dark, unmapped field of ice? The true conflict between our two positions lies in a dangerous paradox of human psychology. Position A shows that we use "P(Doom)" as an **ego-driven narrative engine**—a way to turn abstract technological anxieties into a grand, apocalyptic story starring ourselves as either gods or martyrs. Position B reveals that we use precise percentages as a **cognitive sedative**—a mathematical security blanket designed to tame chaotic, non-linear systems like climate change and high-finance. The friction between them is profound: Are these catastrophic percentages an arrogant, hype-generating attempt to puff ourselves up, or are they a desperate, head-in-the-sand attempt to quiet our deepest fears? ``` [ Complex, Unmapped Systems ] / \ / \ Position A: Ego & Narrative Position B: Fear & Control (AI "P(Doom)" as story) (Finance/Climate as math) \ / \ / [ False Certainty Mirage ] ``` ## The Unspoken Overlap: The Tyranny of "Mathiness" While these perspectives seem to diagnose different social ills, they secretly share a single, corrosive diagnosis: the trap of **mathiness**. Coined by Nobel laureate economist [Paul Romer](https://en.wikipedia.org/wiki/Paul_Romer), "mathiness" refers to using the appearance of mathematical rigor to disguise ideological or subjective claims. Both AI doomers and Wall Street analysts suffer from the exact same statistical illusion. They mistake a subjective guess for an objective measurement. Whether calculated in a Silicon Valley forum or a Manhattan board room, an imaginary probability creates a toxic distraction. It pulls our focus away from present, observable harms—such as carbon emissions today or immediate algorithmic bias—and traps us in abstract debates over hypothetical futures. Statistician [Nassim Nicholas Taleb](https://en.wikipedia.org/wiki/Nassim_Nicholas_Taleb) diagnosed this exact intellectual failure in his book [*Fooled by Randomness*](https://en.wikipedia.org/wiki/Fooled_by_Randomness): > "Probability is not a mere computation of odds on a dice table; it is the acceptance of the lack of certainty in our knowledge and the development of methods for dealing with our ignorance." When we treat probability as a hard prediction rather than a confession of ignorance, we fail Taleb's test completely. ## A Unified Framework: Pragmatic Resilience To move beyond both false hype and false security, we must fuse these insights into a new paradigm: **Pragmatic Resilience**. Instead of trying to calculate the *probability* of an unprecedented disaster—which is mathematically impossible in complex systems—we must measure our **vulnerability** and build systems that can withstand shock. ``` +-----------------------------------------------------------------+ | PRAGMATIC RESILIENCE | +-----------------------------------------------------------------+ | 1. Abandon Map-Making: Stop calculating odds for black swans. | | 2. Audit Vulnerabilities: Identify fragile single points. | | 3. Build Deep Redundancy: Target immediate, observable harms. | +-----------------------------------------------------------------+ ``` This framework replaces the illusion of precise foresight with three actionable principles: 1. **Acknowledge the Blindspot:** Accept that for unique, hyper-complex events—whether an artificial general intelligence, a novel climate tipping point, or a global financial instrument—the true probability is fundamentally uncalculable. 2. **Shift from Risk to Fragility:** Stop asking *"What are the odds of doom?"* Start asking *"How fragile is our system if things go wrong?"* A bank that requires an accurate probability model to survive is inherently fragile. An AI system whose safety relies on predicting its future intelligence is inherently dangerous. 3. **Address Present Vulnerabilities:** Pivot our resources toward tangible, immediate safeguards. In climate policy, this means building resilient infrastructure today. In AI safety, as computer scientist [Timnit Gebru](https://en.wikipedia.org/wiki/Timnit_Gebru) argues, it means addressing the immediate, measurable impacts of data exploitation and algorithmic bias right now, rather than obsessing over sci-fi apocalypses. By stripping away the false comfort of "mathiness" and the ego of apocalyptic storytelling, Pragmatic Resilience forces us to stop predicting the unpredictable, and start preparing for it.

Continue this thread

This path ends here for now.

If you want to keep exploring this line of thought, open the editor and add the next question or answer from this endpoint.

Continue this thread in the editor on desktop.

Other paths you could read

Earlier, at The Siren Song of the Apocalypse: Why AI Was Targeted for P(Doom), the conversation split. If this is not the thread you want, you can switch to the other path below.

Reading key

Highlights

Highlights

15

Notes

0

Links

4