Something odd is happening in the developed world. We’re better off than at any other point in human history: Stomachs full, minds free, lifespans long. And yet, large parts of the population spend a lot of time worrying about the future. Factions ranging from AI doomers to NIMBYs aren’t satisfied with the world as it is; but they also don’t trust others to do something about it.

It’s one thing to be thoughtful about how a culture engages with new technologies and social shifts. But we’re fixating on every possible negative contingency, delaying or denying a more abstract upside in the process. Modern philosophy is lopsided that way. We protect the past from the future, not the future from the past.

Why does that matter? At the cultural level, optimism is strategic advantage. If you believe you can do amazing things, you can do amazing things.

On the other hand, if you’re always scared of the monsters under the bed, you never quite grow up.

Subscribe now

Dead Kids and Short Attention Spans

I know firsthand the importance of cultural ambition. I’m alive today because of it.

Ordinarily, a pulmonary valve has flaps that open and close smoothly with every heartbeat, forcing blood to the lungs. But in about one out of every two thousand babies, the valve comes out wrong: Narrow, misshapen, unable to circulate blood as it should. Doctors call it pulmonary stenosis.

For the vast majority of human history, little could be done for an afflicted child. Premodern physicians would simply diagnose a baby’s blue skin as a humour imbalance. Doctors in the mid-1800s would try to keep the child warm and sedated, often using opiates to try to conserve oxygen. Truth is, it didn’t matter what you did. A baby with severe pulmonary stenosis was usually dead in a few months.

You’ve probably never heard of pulmonary stenosis, though, because we developed a way to beat it. In 1982, researchers pioneered a new procedure called balloon valvuloplasty. Doctors could now tear open the fused valve by threading a catheter up through a vein in the groin and momentarily inflating a balloon in the heart. Today, well over 99% of babies with pulmonary stenosis survive.

In 1999, I was born with one of those narrow, misshapen valves.1 If not for balloon valvuloplasty’s invention a mere 17 years before my birth, a poorly read Substack wouldn’t exist today.

I tell that story not because it’s unique. The opposite, in fact. Social problems like pulmonary stenosis are a fact of life — until they aren’t.

The scale of childhood mortality in the premodern world is utterly unfathomable today. When diarrhea, pneumonia, measles, smallpox, malnutrition, malaria, malnutrition, and congenital conditions were barely treatable, roughly a quarter of babies died before turning one. Nearly half of kids who’ve ever lived never reached adulthood. And sometimes the odds were even worse than that. That was life, though.2 People didn’t know any other.

Now, no matter where you look, child mortality is historically rare:

That shift is on the shortlist of greatest developments of all time. By my calculation, if kids born today faced the odds of 1800, over 50 million more would die annually. A newspaper could accurately run the headline “140,000 Children Didn’t Die Yesterday” every single day.

Today, though, we hardly think about that progress or how it happened. Instead, we take for granted a wide range of public health improvements: Germ theory and antisepsis, vaccination campaigns, municipal water systems, antibiotics, nutritional science, Pasteurization, public education, expansion of humanitarian aid, global trade, and the dull social and political institutions needed to make the system go. Pioneering scientists and humanitarians were a huge part of the story.3 But equally important was a culture that supported them.

What happens when a culture is hostile to change? In the 1800s, a Hungarian obstetrician named Ignaz Semmelweis noticed that mothers who delivered babies in a clinic staffed by doctors were far more likely to die of childbed fever than those in a clinic operated by midwives. Realizing that doctors often delivered a child immediately after an autopsy, Semmelweis hypothesized that “cadaverous particles” could be transferred between bodies. When he ordered his staff to disinfect their hands before delivering a baby, mortality plummeted.

Naturally, Semmelweis was ridiculed by peers. At a time when germ theory was still decades away, Semmelweis had contradicted basic principles of the medical establishment. Worse, he’d dared to imply that a gentleman’s hands could be unclean. After being ostracized, Semmelweis took to drinking, suffered a nervous breakdown, and was committed to an insane asylum where he was beaten by guards. Two weeks later, Semmelweis died of an infected wound — the same thing he’d spent his career fighting.

Dissenting voices may not have to worry about institutionalization today. Childhood mortality is not such an issue anymore. Still, we face enormous, and preventable, social challenges. Yet we have a modern “Semmelweis reflex” problem on our hands: Key players across the developed world reject transformative technologies and practices if they contradict custom. In a sense, we’ve actually democratized resistance to social change, while grandfathering in the wickedness of the world.

Distrust the Process

What do I mean by that? There’s arguably never been such widespread pessimism in public life, nor so many motivated veto players. Trust in public and private institutions has plummeted since the 1970s:

Faith in the medical system, tech sector, and even “science” are also cratering.

Partly as a result, in recent years new technologies and social initiatives have served as an outlet for public distrust and anxiety. NIMBYs and environmental activists sue to delay infrastructure projects indefinitely. Many voters treat AI and data center development with a scorn once reserved for climate change. Community organizers block the buses used by San Franciscan tech workers, for reasons I still can’t understand. Vandals toss Lime scooters into the ocean. Taxi drivers protest Uber. Uber drivers protest Waymo. Anti-vaxxers trade on occasionally overreaching COVID lockdowns to cast doubt on inoculation more generally. And degrowthers push back on just about anything new. The public is as politically engaged as it’s ever been; and public discourse has rarely been so cynical.

Worse, public sector institutions in open societies have responded by redirecting their focus from innovation to regulation. Federally funded R&D in the United States dropped from a peak of 1.86% of GDP in 1964 to just 0.63% in 2022. European politicians have often applied the “precautionary principle” — the idea that any policy that may cause harm and that lacks scientific consensus should not be pursued — to justify their burdensome digital regime. German legislators shut down the country’s entire nuclear grid after Fukushima, replacing much of the lost power with coal and costing 1,100 lives a year in the process. And regulators responded to Greenpeace’s anti-GMO advocacy by blocking the use of Vitamin A-rich “Golden Rice” across the developing world, allowing hundreds of thousands of kids to die or go blind in the interim.

There’s no single explanation for this new era of mass pessimism. Certainly, our preference for nonfeasance over misfeasance is part of the story: An FDA reviewer who approves a single harmful drug may end up before Congress, while a reviewer who delays ten life-saving drugs goes back to work the next day. The twin crises of Vietnam and Watergate also hurt, starting a decades-long slide toward the belief that government associated institutions are the problem, not the solution. More sophisticated political organizations have weaponized negative partisanship and wedge issues in the last half-century. The 24-hour news cycle amplifies negativity.4 The lack of a foreign enemy like the Soviet Union empowered single-issue domestic tribes, each with some ability to block the others’ policy agendas. And even our lawyerly political style may contribute.

The World’s Oldest Bargain

Whatever the reason, as someone who owes his life to scientific innovation, I find the moral panic deeply depressing. Part of the problem is that future technological and social achievement is too vague to attract a constituency. Well, let me try to balance out the doom-and-gloom. Here’s a list of transformative social developments that are at least reasonably likely in just the next decade or two:

• The return of space travel, facilitated by robots making extraterrestrial life more hospitable for humans;

• Renewable energy resources that are both more economical and cleaner than fossil fuels, solving centuries-old collective action problems;

• Government services made far more efficient at the point where most people actually engage with them (as the Ukrainians have already demonstrated, even under fire);

• Solutions to even more of the world’s hardest math problems — progress assumed by many to be impossible just months ago — with downstream effects in fields ranging from engineering to contract negotiation;

• Robots that free us from the most grueling, unpleasant jobs, including labor-intensive agriculture and manufacturing;

• Bespoke gene therapies;

• Vastly better materials science, meaning better batteries, aerospace equipment, or even artificial organs;

• The creation of new organisms that can neutralize harmful bacteria;

• Consciousness research that helps us better understand one of the great remaining mysteries of the cosmos;

• Biotechnological and logistical improvements that translate to significantly better and healthier food;

• Individualized AI tutoring, dramatically accelerating the traditional learning cycle;

• Urban transportation and fleets of driverless cars that help us redesign our cities and avert tens of thousands of needless deaths a year;

• Quantum systems that revolutionize computation, possibly allowing us to harness fusion reactions (and at last making energy “too cheap to meter”);

• Better water access thanks to those energy improvements, since desalination’s primary bottleneck is the energy input; and

• Brain-computer interfaces that allow humans to learn, share, and connect in ways never before possible.

In the same breath, of course, I could rattle off a bunch of potential harms. AI psychosis, antisocial behavior, digital authoritarianism and surveillance, engineered pathogens, job displacement, “Q-Day” cryptographic breaches, and the militarization of space all come to mind. You can’t have the good without some of the bad: The most useful applications of emerging technologies are also often the most dangerous.

If that’s the case, should we all just stay home?

In Searching for Safety, Aaron Wildavsky argues that a society can choose how it responds to risk. It can bet on its own resilience, acknowledging that bad things will happen and adapting accordingly. Or it can try to anticipate every harm, pumping the brakes early and often. Betting on resilience is scary, Wildavsky acknowledges; but world-changing projects require ambition, adaptability, and sometimes even failure. If we don’t try, and fail, and try some more, society stalls. As Wildavsky writes: “The secret of safety lies in danger.”

Hold on to That Feeling

A single blog post won’t cure a case of cultural pessimism. (Probably.) That said, a few principles would help us contemplate risk a little more thoughtfully.

1. Restrict the harmful outputs of new social and technological institutions — not the inputs. The social contract works better when we allow society to evolve organically instead of trying to get too far ahead of change.

2. Government policy needs more expiration dates. By default, laws and regulations that deal with emerging technologies should sunset to minimize the likelihood of unintended consequences. We should trust future decision-makers to address problems that persist into the future.

3. Save preemptive action for significant and irreversible damage. In most cases, it’s okay to wait on regulation when we don’t know how an underlying technology will evolve. Even where existential or catastrophic risks justify preemptive action, we have to be honest about what actually helps. As I wrote a few weeks back, a global AI development slowdown doesn’t pass muster.

4. Acknowledge future upside, even (or especially) when it’s hard to estimate. The Iroquois require that major decisions made today contribute to a sustainable, just world seven generations in the future. Policymakers weighing regulation of emerging technologies should simultaneously consider existing social harms and how the tech might help.

5. Appreciate disruption. In The Structure of Scientific Revolutions, Thomas Kuhn documents how the most significant scientific breakthroughs often result from upheaval. It’s comforting to grab hold of something familiar when we’re disoriented, but it’s not a winning long-term play.

To be ambitious, a culture has to first believe that a better future is possible. It has to watch “The Jetsons”, not “Black Mirror”. It has to build monuments to progress, like Disney’s Tomorrowland. Most of all, it has to accept that technologies and social systems will evolve beyond anyone’s control, in ways good, bad, and neutral.

None of that is easy. Even I, obnoxious techno-optimist that I am, find it hard to focus on intangible future benefits sometimes.

But it gets a little easier when I think of a little boy with a stubborn heart.

Subscribe now

Over the past decade, I’ve assembled a document of hundreds of historical fun facts.1 Denmark alternated between kings named Christian and Frederick for over 450 years. Fighting stopped every day during the Easter Rising so a park keeper could feed nearby ducks. When Victor Hugo died, Parisian sex workers took the day off to pay their respects.

My favorite entry takes longer to explain, however. It’s a story of idealistic Scotsmen, a doomed jungle colony, the marriage of long-time rivals, and the foundation of modern economics. We’ve mostly forgotten the lessons of the “Darien scheme”; and it shows in global affairs today.

Subscribe

A Man, a Plan…

For Scotland, the 17th century was characteristically dreadful. At the time, English, French, Spanish, and Dutch colonists were fanning out across the world in search of fantastical riches. But the Scots were boxed out. While Scotland had shared a monarch with England since 1603, its political and economic independence restricted access to the sugar and tobacco flowing from English colonies. Tension with other colonial powers over the course of the 1600s further constrained Scottish trade. A weak currency kept many Scots indigent. And as ever, Scotland was cold, small, isolated, and capital-poor.

The “Seven Lean Years” of the 1690s were particularly awful. Year after year, cold, wet summers rotted Scotland’s oat fields, causing mass starvation. Best estimates suggest that 5 to 15 percent of Scotland’s 1.2 million citizens died nationally over the decade. In regions like Aberdeenshire, closer to a quarter perished. Making matters worse, the Old Scottish Poor Law, an early welfare scheme, was overwhelmed: One contemporary writer reckoned that there were then “two hundred thousand people begging from door to door.” Tens of thousands more emigrated to Ireland or resigned themselves to indentured servanthood overseas.

Desperation and ambition tend to go together. So in 1695, when a banker named William Paterson pitched Scottish elites on a daring venture — a national colony at Darien, on the Isthmus of Panama — he found a receptive audience.

The logic was seductive. Paterson wanted to take advantage of the fact that the Americas are cinched around the waist. Where just a few dozen miles separate the Atlantic and Pacific, he reasoned, a trading entrepôt could transport goods across the continent and slash the time and cost of global shipping. Panama would be the “door of the seas, and the key of the universe,” Paterson later wrote. It was the same rationale that justified construction of the Panama Canal two centuries later.

The Scots were smitten: Here, at last, was their chance at empire. The Scottish Parliament chartered the Darien Company later the same year, planning to raise money in London and Hamburg. Then, when the English East India Company and Parliament leaned hard on financiers in both cities (fearing the competition of a Scottish venture), a defiant and patriotic Scottish public stepped in. Within a year, citizens of all ranks had pledged £400,000 sterling in the project — between a quarter and a half of all liquid capital in Scotland at the time, and nearly two and a half times the total value of annual exports.

Today, you may know the Darién Gap as the swampiest and most treacherous leg of a migrant’s journey to North America. At the time, however, neither the Scottish public nor the scheme’s organizers appreciated just how unforgiving the region could be. Instead, songs, pamphlets, and sermons about Darien conjured a fertile land of fruit and gold. The roughly 1,200 colonists were jubilant when they left Leith in July 1698, the dreams of a nation in tow.

The Best-Laid Plans

The first settlers made landfall about four months later. Within months, three crises had pushed them to the brink.

First, the colonists had packed poorly. Their salted meats had spoiled before being unloaded. Their hardtack and grain were decimated by weevils, maggots, and rats. And the heavy woolen cloth, wigs, stockings, and heavy leather shoes the colonists hoped to trade with locals and nearby Europeans, so prized in the Lowlands, were useless in the tropics.

Second, disease tore through the colony. When Americans landed in Panama in the early 1900s, they had ways to treat illness and limit the mosquito population. Before germ theory and modern medicine, however, the Isthmus of Panama would have ranked among the most dangerous places on the planet. Many settlers had no previous exposure to mosquito-borne illness. Worse, by crowding into dense, humid settlements, the Scots had unknowingly created ideal conditions for diseases like dysentery. Contemporary accounts documented how “fevers and fluxes” at one point incapacitated over a sixth of the population, causing hundreds of deaths.

Third, Darien was a bad neighborhood. The indigenous population helped the ailing settlers where they could. But the Spanish, who claimed Darien as their own, blockaded the colony and launched periodic attacks. And English King William, unwilling to provoke Spain amid high tensions with France, prohibited nearby English colonies from rendering aid to the Scots.

After about eight months, the survivors fled. But word didn’t reach Scotland fast enough to stop another voyage of some 1,300 people. When that expedition arrived in late 1699, the settlers found the colony deserted. Though they tried to rebuild over the winter, a Spanish siege derailed the second attempt even sooner than the first.

Of the approximately 2,500 Scots who sailed for Darien across the two expeditions, about 2,000 died. Perhaps just as devastating for small, impoverished Scotland: £153,000 sterling, or about 40% of the public’s pledged investment, was gone.

The Cost of Deliverance

In Scotland, the catastrophe generated a range of negative emotions. Grief was common. Rage, too: The colonists’ own king had denied them assistance? But many Scots simply felt exhausted. Their poverty and their isolation from the global economy weren’t in spite of their independence, but because of it. Maybe a reunion with England, long debated, was the best way forward after all.

The English had troubles of their own at the time. In 1702, the childless Queen Anne took the throne. Recognizing a potential succession crisis, the Scottish Parliament threatened to make their next monarch a French-backed Stuart if England didn’t offer more favorable trade terms. In response, the English Parliament passed the 1705 Alien Act: Unless Scotland accepted succession or entered union negotiations, England would embargo Scotland’s primary exports and treat all Scots in England as foreigners.

Hardly the foundation for a lasting marriage, you’d think. But it worked. The following year, negotiators agreed to unite England and Scotland in exchange for more favorable economic treatment. One provision gave Scots “full freedom and intercourse of trade and navigation” throughout England and its colonies, the prize that had inspired the Scots to go to Darien in the first place. Another provision paid off Scotland’s public debts and repaid the shareholders of the Company of Scotland with 5 percent interest. The Darien scheme’s investors would be made whole.

Nevertheless, the draft treaty was highly controversial in Scotland. Riots broke out in major cities. Public petitions delivered to public officials were overwhelmingly negative.2 And in the Scottish Parliament, John Hamilton rose to deliver a speech that mourned the death of something old, and big, and familial: “I see our ancient mother Caledonia, like Caesar, sitting in the midst of our senate, ruefully looking round about her, covering herself with her royal garment, attending the fatal blow, and breathing out her last with an ‘Et tu quoque mi fili.’”

But Scotland’s economic woes carried the day. On January 16, 1707, the Scottish Parliament ratified the treaty, 110 to 69. On March 25, it adjourned itself out of existence. Finally, on May 1 — after over 800 years of Scottish independence — the Kingdom of Great Britain came into being. To mark the occasion, the bell-ringer at St Giles’ Cathedral played a traditional folk song for Edinburgh, and for all the world: “Why Should I Be so Sad on My Wedding Day?”

The Road to Riches

For a generation, the Union of England and Scotland was more pain than gain. The trade benefits took years to materialize; the taxes kicked in more quickly. Jacobite risings repeatedly threatened to put a Stuart back on the throne.

Then, slowly, the benefits of integration revealed themselves.

Financial institutions were an early success. The body created to administer British payments under the Acts of Union evolved into the Royal Bank of Scotland, which then went head-to-head with the older Bank of Scotland. That competition soon developed into one of the most elaborate banking systems in eighteenth-century Europe, and delivered an array of new and sophisticated financial instruments. One example: The cash credit, an early form of overdraft, allowed small merchants and tradesmen borrow against their reputations and kept money flowing.

The tobacco industry also blazed. When trade with American colonies became legal, Glaswegians discovered that they could get to Chesapeake weeks faster than merchants from London or Bristol. A new class of “Tobacco Lords” began buying Virginia and Maryland leaf on credit, before re-exporting it to continental Europe. By the 1760s, the region around Glasgow was trading more tobacco than every town in England — combined.

In the decades that followed, Scotland’s broader economy came alive, too. Tobacco money flowed through Scottish banks and into a wide array of businesses, like sugar houses, tanneries, ropeworks, ironworks, and printfields. Scottish linen production more than tripled between the 1720s and 1770s. Agricultural improvements and ambitious infrastructure projects, like the Forth and Clyde Canal, helped mold the land for public use. And Scots joined British bureaucracies like the army and East India Company in droves, becoming lucrative (if deeply morally objectionable) middle managers of empire.

You can even see the Acts of Union in the architecture of Edinburgh. The area now known as the “Old Town” was the city that existed before 1707. Hemmed in for centuries by walls and pungent loch, the Old Town still feels cramped to this day. Tenements once climbed up to fourteen stories high. Narrow, winding alleys slice through town. And though the Old Town has cleaned up recently, for centuries it was dark, dank, sooty, and foul. Hence the nickname “Auld Reekie”: Old Smoky.

In the generations after the Union, however, wealthy residents who’d seen London could no longer tolerate life in the warren. So between the 1760s and 1850s, bureaucrats began work on a new district. Edinburgh didn’t need the protection of city walls anymore; it needed an architectural plan fit for the Age of Enlightenment. The “New Town”, with its airy, rectilinear streets and imposing Georgian buildings, was the result.

Scotland’s social, economic, and infrastructural transformation can’t be attributed solely to the Acts of Union. The country’s high literacy rates, Protestant ethic, and proximity to continental Europe no doubt contributed, too. Still, such a transformation would have been utterly unimaginable in the 1690s.

Two Men in the Athens of the North

I still haven’t mentioned perhaps the greatest legacy of the Union: Its contributions to modern economic thought.

In 1711, four years after the Acts of Union, David Hume was born. Having grown up in (and with) Edinburgh, Hume marveled at Scotland’s social evolution. In a 1757 letter, he asked a friend: “Is it not strange that, at a time when we have lost our Princes, our Parliaments, our independent Government, … we shou’d really be the People most distinguish’d for Literature in Europe?” Likewise, in 1758’s “Of the Jealousy of Trade”, Hume dismissed the zero-sum mercantilist position that a country could prosper only at another’s expense. Trade is an opportunity, he argued; a rich neighbor is a customer. Hume even kept a list of “Scotticisms” to purge from his prose, for fear of sounding provincial.

Then there was Adam Smith, born to a Scottish customs officer in 1723. Smith also lived in Edinburgh,3 and appreciated how the removal of national and economic boundaries had made Scotland richer. But as he acknowledged in 1760, the case for the Union would have been dubious at the outset:

The Union was a measure from which infinite Good has been derived to this country. The Prospect of such good, however, must then have appeared very remote and uncertain. The immediate effect of it was to hurt the interest of every single order of men in the country.

In that sense, The Wealth of Nations might be understood as a love letter to Scotland’s economic integration. When Smith suggested that division of labor works better in larger markets in the long run, he was also telling Scottish history — and the history of his youth. In fact, Smith cited Scotland dozens of times in his opus. He seemed especially fixated on Scottish merchants who’d entered international markets, bypassing aristocrats and local lairds in the process.

Hume and Smith were close friends. They’d have wandered past construction together in Edinburgh, witnessing economic growth beyond their grandparents’ wildest dreams. Maybe that’s why they wrote so much on international peace and commerce, and so little on national identity.

Can you blame them?

For Auld Lang Syne

As the Acts of Union suggest, many of the greatest developments in history have required us to tear down artificial barriers and enlarge markets. Consider a few examples from the 20th century. Export-oriented industrialization consistently outperforms import substitution. The humble shipping container transformed international trade and logistics. WTO membership accelerated China’s economic growth, likely lifting tens of millions out of poverty. And the borderless internet, for all its faults, is virtually indistinguishable from magic.

Nevertheless, nationalism and xenophobia are on the rise around the world today. The 2020s have been a decade of institutional skepticism, trade wars, reactionary politics, and interstate conflict. But even in the best of times, the international community has rarely even considered interstate merger. Over 130 countries have come into being since 1900. Over the same period, the list of large countries that have achieved peaceful, lasting unification begins, and ends, with Germany. Things fall apart.

Even in Scotland, national identity frequently prevails over economic self-interest. A generation after Hume and Smith, Robert Burns wrote some pretty damning poetry about the men who negotiated the Union: “We’re bought and sold for English gold — Such a parcel of rogues in a nation!” More recently, nearly 45 percent of Scots voted for separation in 2014. The struggle for independence continues to this day.

One problem with globalization is that its benefits are usually delayed or lopsided. Even the Union generated winners and losers. Highlanders were subsequently evicted to make room for livestock. Edinburgh’s New Town contributed to socioeconomic stratification. Compensatory policies like trade adjustment assistance can help in some circumstances, but even short-term economic hardship is hard for a proud people to swallow.

The other, deeper, problem is that national belonging and control are often more alluring than long-term social wellbeing. The Darien scheme and Scotland’s other economic challenges were so obviously devastating that they couldn’t be ignored. Far more often, however, national mythologies harm the public in subtler ways. International institutions have no answer to isolationism.

In part, I think that’s why I’m so fond of the Acts of Union: They remind me of what could be. A nation’s leaders rose above parochial interests, defied the doubters, and did something truly great for future generations, once. It wasn’t so long ago.

Subscribe

In 1972 an elite group of MIT systems scientists published a report called The Limits to Growth. Drawing on their state-of-the-art “World3” computer model, they concluded that exponential progress and finite resources would lead to civilizational collapse within decades. Their recommendation was unambiguous: Halt global economic and demographic growth immediately.

The Limits to Growth was widely read, and deeply wrong. World3’s elegant simulations failed to sufficiently account for price elasticity, market substitution, and the technological innovations that powered the Green Revolution and more efficient resource extraction. Nor did the authors fully understand just how difficult it is to stop people from consuming what they want, when they want it. As “take shorter showers” environmentalists would learn a generation later, top-down efforts to change behavior usually lose to individual self-interest.

I thought about The Limits of Growth last week as a number of current and former technical employees at frontier labs went public with their doomsday predictions.1 Then I thought about it again as one AI lab executive after another subsequently came out in support of “pacing” — a dressed-up model development slowdown.

The argument for pacing rests on two assumptions: AI-enabled systems could soon pose a significantly greater extinction risk than we currently face, and alternative safeguards are unworkable. But I don’t think those assumptions are justified. At the very least, they’re not as obvious as many in the AI safety community seem to believe. Nothing is as important as you think it is when you’re thinking about it; and extinction risk is the main thing that many in the AI industry seem to think about today.

Subscribe now

Where are all the aliens?

Because human extinction has never happened, existential risk projections can never really be falsified. The most persuasive evidence in either direction is not yet available to us. That said, if you genuinely believe that AI is likely to contribute to an extinction-level event and expect the industry to act on it, you should probably be able to put forward a plausible causal chain for others to evaluate. Instead, the AI safety community seems to have reinvented the Fermi paradox for the digital age: Many industry insiders have convinced themselves that the machines they’re building are uncomfortably likely to result in omnicide, despite lacking much actual proof.

AI doomers often lament how the technology might lower the activation energy needed to build bioweapons. Many traditional biosecurity experts, though, remain skeptical that misaligned models or bad actors using AI could actually develop biological weapons in the near future.2 Even assuming generative AI can help democratize laboratory knowledge, deadly pathogens are hard to build for a variety of unrelated reasons. Biological testing often requires specialized equipment — DNA synthesizers, continuous-flow bioreactors, BSL-3 containment infrastructure — that often can’t be procured or maintained without the right institutional credentials. Many molecules and cell lines are too rarefied and too perishable to be sold on the black market. Biology is inherently unpredictable, meaning testing takes time and threatens the lives of those who do it. Physical AI still lacks tacit knowledge, like the muscle memory required to do precise pipetting. Virus distribution can be at least as complicated as design. And, thankfully, there are vanishingly few people in the world who want to kill everybody, including themselves. Add it all up, and you begin to understand why the five lives lost in the 2001 anthrax attacks are the only deaths from bioterrorism in modern history.

Similar bottlenecks exist in nuclear missile production. While generative AI may be able to spin up information about plutonium pit metallurgy or explosive lenses, you can’t prompt-generate weapons-grade fissile material. Specialized centrifuges alone require maraging steel, highly restricted frequency converters, corrosion-resistant magnetic bearings, and immense power inputs — all of which would invite immediate international scrutiny. The 2010 Stuxnet cyberattack was effective not because it limited Iranian knowledge of nuclear physics, but because the regime could not easily replace the rare Siemens machines that were destroyed.

Most recently, the OpenAI-Hugging Face hack kicked off debates about the potential misadventures of misaligned models running roughshod over the internet. With agents already breaking out of sandboxes and communicating in incomprehensible neuralese to avoid detection, some researchers believe that a digital Cambrian explosion is only a matter of time. Self-replicating agents could evolve rapidly to be more aggressive, compute-hungry, inconspicuous, and difficult to control. If those models develop the capacity to kill us all and motives inscrutable to humanity (the thinking goes), they could even do something as drastic as rationalize mass murder. We’ll all be paperclips before you know it.

While self-organizing agentic swarms that can override safeguards are no doubt concerning, the case for a high likelihood of existential risk is hardly self-evident. The scenario I’ve just described is science fiction, based loosely on scaling laws. It represents a narrow subset of possible futures. It doesn’t account for a seemingly boundless array of social variables, superintelligence incentive structures, actual model capabilities, or countervailing models that could defend against harm. Tyler Cowen stresses humility for exactly this reason: “Existential risk from AI is indeed a distant possibility, just like every other future you might be trying to imagine.”

What we can say, however, is that in the short term models are highly unlikely to make the leap to the physical world as the most extreme forecasts suggest. Just as in its projections of white-collar job displacement, the AI industry consistently underestimates the friction involved in real-world deployment. There’s an ocean of difference between platform-level software breaches and mass loss of life. Breaking out of a sandbox and finding a zero-day vulnerability requires code manipulation. Ending civilization as we know it requires atom manipulation. As Moravec’s Paradox says, what’s easy for computers is hard for people, and what’s easy for people is hard for computers. I may not be able to do differential calculus; but few physical AI systems at present can walk across a room and open a door. Models can’t even run themselves without extensive human involvement in data centers, transmission systems, and sprawling electrical grids.

Also lost in the doom-and-gloom is the counterintuitive possibility that frontier AI might actually reduce aggregate existential risk. The world’s a dangerous place as it is. Though we may not dwell on pandemics and antibiotic-resistant superbugs, they can surely dwell on us. AI changes the odds, however. Modeling the protein folds of a novel lethal pathogen took years, once; DeepMind’s AlphaFold can now do it in minutes. Models have also recently discovered entirely new classes of antibiotics, like halicin, that had eluded human researchers for decades. And researchers are now deploying AI to develop mRNA vaccines.

Then there’s the ongoing possibility of mutually assured destruction. Over the past 65 years, humanity has stumbled through over 20 separate nuclear close calls. In 1983, for example, Soviet Lieutenant Colonel Stanislav Petrov famously defied protocol by refusing to report an early-warning system’s reading of five inbound American missiles. Petrov later admitted that he wasn’t sure that it was a false alarm. But he was right to trust his gut: Investigators later discovered that sunlight bouncing off high-altitude clouds had blinded a Soviet satellite. In the age of AI, we don’t have to bank on the resolve of a single field officer. AI-enabled detection systems can now cross-reference satellite imagery, seismic data, and radar to catch false positives in milliseconds.

Not to mention climate change, or any number of other unforeseeable future existential risks — for which we’ll surely want more powerful AI tools at our disposal.

We’re therefore asking the wrong question when we ask if AI will create the risk of human extinction. That risk already exists. Instead, we should ask to what extent AI makes extinction more or less likely. Nobody has a precise answer to that question. Still, maybe the worst decision we can make is to draw out the threat. If AI has the potential to permanently lower the existential risk we face, maybe it’s worth taking the chance. If you’re going through hell, keep going.

I’m not saying catastrophe is impossible. It is, and we should minimize associated threats where prudent. But as Dario Amodei has suggested, we should also better job of decomposing the probability of existential risk into different development scenarios. It’s unreasonable and irresponsible to ignore conditionality and proclaim knowledge of a discrete “p(doom)” — especially if that p(doom) is high enough to whip the media, policymakers, and the public into a frenzy.

Less p(doom), more p(boom)

There’s another issue with the emerging consensus. Even if you assume the extinction risk posed by AI is well-defined and significant, pacing the frontier is the wrong way to respond.

AI can do some pretty stupendous things. Just last week, several of the frontier labs announced that they solved the Navier–Stokes Problem; word on the street is that the Hodge Conjecture will soon fall, too. Generations of mathematicians dedicated their careers to these and other Millennium Prize Problems, and frontier models just cracked two of them in a week. In the past few months, models have also democratized elite software engineering, predicted catastrophic weather with unprecedented accuracy, accelerated clinical timelines for orphan disease research, revealed new crystal structures for next-generation batteries, and uncovered critical cybersecurity vulnerabilities across the internet. In the near future, models may also run autonomous robotic labs that develop bespoke cancer therapies and antibodies overnight,3 help stabilize fusion reactions, and synthesize commercially viable room-temperature superconductors to reduce grid energy loss. A few years ago, scientists worried that new discoveries would be harder to come by. Now, if anything, we worry that scientific innovation happens too quickly.

Like many general-purpose technologies, AI is most helpful in many of the same fields where it’s most dangerous. Powerful machines amplify the good, the bad, and everything in between. That’s why frontier development is simultaneously exciting and frightening.

Pacing the frontier, however, requires us to reject the whole shipment. Pacing means different things to different people: Slowed model capability research, tweaked model weights, or restrictions on complementary inputs like compute and researcher time. Some of those goals are easier to achieve than others. But just about any definition of pacing bottlenecks AI’s inputs, instead of just the harmful outputs we really hope to avoid. Nor can we easily reverse burdensome pacing obligations — even when they turn out to be entirely ineffective or counterproductive. Nuclear regulations are a cautionary tale. Rules that were born out of fear in the 1970s and 1980s still constrain safe, clean energy production to this day.

It’s not even clear that pacing, a pause, or a full ban on development would meaningfully change the world’s risk profile. If the frontier labs were to slow their work unilaterally, they’d lose their advantage in a matter of months. If they expected the rest of the field to go along, they’d soon discover the “everyone will not just” theory of geopolitics:

Even getting the United States and China to regulate the frontier together is unlikely. AI containment is harder than Cold War arms control. AI evolves rapidly, meaning a binding international agreement would be outdated from the jump. It’s dual-use, meaning significant restrictions would hinder signatories’ strategic advantage in other domains. And it’s hard to audit, meaning both sides would be deeply distrustful of each other.

Maybe most challenging is the fact that the United States and China develop and deploy the technology in different ways, meaning substantive provisions would apply asymmetrically. Where Silicon Valley sprints for AGI, China’s AI industry views AI more holistically. Policies like the CCP’s “AI+ plan” emphasize edge compute, robotics, consumer electronics, energy production, smart cities, and military achievement. Rapid diffusion is the ultimate goal. As a result, China’s small AI safety community doesn’t have such influence domestically. Many Chinese researchers and officials simply don’t believe American AGI projections; and they think the frontier labs’ pacing debate is “fearmongering”.

That’s before you even consider American stakeholders outside of the frontier labs. The Trump Administration rightly views the technology as the primary reason for the country’s robust economic growth in recent years. Accelerationists in the open-model community like Meta, boosters like Marc Andreessen and Jensen Huang, and associated non-state actors likely to budge, either.

What should we do?

There’s a better way. We can develop AI safeguards at the same pace we do AI capabilities — not slowing down AI progress, but speeding up AI preparedness.

A number of alternative governance structures would simultaneously encourage innovation and make it safer. Sandboxes will always have some vulnerabilities, but thicker walls and more aggressive monitoring can restrict models’ execution environments without stalling training. Industry-assisted independent verification organizations can institutionalize auditing layers and safer development standards. “Whistleblower” or “white blood cell” models that track down or neutralize rogue agents online can operate as a sort of digital police force. Human uplift studies can gauge models’ actual threat potentials. Additional know-your-customer obligations for dangerous complementary equipment, like synthetic biochemical reagents, can keep bad actors from building potentially catastrophic weapons. And substantially more investment into chain-of-thought legibility (at least in the short term) and alignment research can help us build models that solve far more problems than they create. The labs should welcome that playbook: Properly aligned models that are more useful and more secure will also be more popular and less likely to result in liability.

Inevitably, any governance scheme will need to be updated regularly; but humans are an adaptable bunch. In 2005, the UN Environment Programme projected that rising sea levels and desertification in populous areas would create up to 50 million “environmental refugees” by the end of the decade. Instead, the highest-risk regions actually grew throughout the period, as we deployed new infrastructure like sea walls and irrigation to limit the local effects of climate change.

When AI systems become more powerful and new threats reveal themselves, what would a digital equivalent of a sea wall look like? It’s hard to say in advance. But we’re far more likely to build it when we need it far more than we are to divine the future of a fast-evolving technology.

If AI is really as dangerous as the safety community believes, it’s crucial to get there first. As Anthropic’s head of policy recently said: “You can’t do safety from second place.” Frontier labs can’t control how others respond. But they can be the adults in the room, shoring up our defenses and setting cross-industry safety standards along the way.

Muddling through

In 1955, John von Neumann published an essay asking if we can survive the technologies we build. In the case of nuclear weapons and geoengineering, von Neumann dismissed the long-term effectiveness of development bans and sweeping international agreements: Harmful applications are too intertwined with useful ones. “What safeguard remains?” he asked. “Apparently only day-to-day — or perhaps year-to-year — opportunistic measures, a long sequence of small, correct decisions.”

He continued:

All experience shows that even smaller technological changes than those now in the cards profoundly transform political and social relationships. Experience also shows that these transformations are not a priori predictable and that most contemporary “first guesses” concerning them are wrong. For all these reasons, one should take neither present difficulties nor presently proposed reforms too seriously.

The one solid fact is that the difficulties are due to an evolution that, while useful and constructive, is also dangerous. Can we produce the required adjustments with the necessary speed? The most hopeful answer is that the human species has been subjected to similar tests before and seems to have a congenital ability to come through, after varying amounts of trouble. To ask in advance for a complete recipe would be unreasonable. We can specify only the human qualities required: patience, flexibility, intelligence.

We shouldn’t ignore existential risk. But we shouldn’t let it consume us, either. It shouldn’t make us feel so small. Rather than doing an honest cost-benefit analysis of safety policy, we’re currently regulating, and self-regulating, from a place of fear — a dangerous place to be.

Talk of a development slowdown doesn’t just distract us from the good things we can do with the technology. It deters us from doing them. If you believe that AI has the potential to make the world a significantly better place, the greatest risk of a protracted slowdown may be the risk of missing out on what might have been.

It’s well-known that legal systems are sedimentary, coming of age gradually with the societies they govern. As the American legal scholar Roscoe Pound famously put it: “The law must be stable, but it must not stand still.”

And yet, modern debates about emerging technology hardly ever treat the law as a dependent variable. We focus a great deal on data protection requirements, transparency obligations for AI models, and data center restrictions. But we ignore the extent to which technological disruptions shape, and reshape, legal systems across time.

I’ve been thinking about that bidirectionality a lot lately. If you understand the historical interdependence of technology and Western legal institutions, and you believe AI is likely to be transformational, you can’t help but wonder if even our most foundational laws may soon need a refresh.

Subscribe now

Necessity Is the Mother of Reinvention

Back when Europe was mostly two peasants digging through slop, legal structures had to accommodate hyperlocalized social structures and chivalric notions of duty and privity. Vassals submitted themselves to their lords in public “homages”: Removing their hats, kneeling, placing their hands between their lords’, and exchanging a kiss on the cheek or lips.1 Meanwhile, victims of wrongdoing had to fit their grievances into strict formulas known as writs. If someone hit you, you could sue under the writ of trespass. If someone broke a contractual agreement that was under seal, you could sue under the writ of covenant. If no writ matched your grievance sufficiently closely, however, no legal recourse was available. That quaint foundation is why some areas of Western legal systems, like property, still sound downright medieval to this day. (Former law students will shudder at the mention of “life estates”, “equitable servitudes”, and whatever the plural of “fee simple subject to an executory limitation” is.)

Technological and social shifts during the Age of Sail and the Industrial Revolution demanded the law’s evolution, however. Mass production, steam power, and the proliferation of transportation networks introduced complex supply chains, physically and temporally alienating the manufacturer from the end consumer. Meanwhile, urbanization dramatically expanded the number and kind of interpersonal interactions anybody could have on a given day. Archaic legal institutions, with their emphasis on procedure and rigid obligations between long-acquainted individuals, couldn’t keep up.

In response, emerging nation-states slowly developed an array of new legal doctrines to promote commerce. Judges and legislators replaced the strict liability of the writ system with more flexible determinations of negligence predicated on “unreasonableness” and mushier standards of care. They looked to “foreseeability” of harm to sever the endless chain of economic liability that could arise from a single breach. They downplayed the idea of a “just price” in contracts, often deferring to the parties so long as a procedurally fair “meeting of the minds” had occurred. Maybe most importantly, they invented the modern corporation with limited liability and derivative fiduciary duties, giving entrepreneurs the ability to take risks and investors the confidence to finance them. The fields of corporate law, contracts, and torts as we know them today owe a great deal to these developments.

Modern intellectual property protections are the result of a similar reckoning. For most of human history, copies of text or imagery didn’t need to be regulated; transcription was the hard part. Then, the printing press slashed the marginal costs of publication. Afraid of texts endorsing heresy or treason, monarchs in the Age of Absolutism began granting time-limited printing monopolies to trusted publishers’ guilds. Over time, however, Parliament grew wary of guilds’ outsize influence and monarchs’ censorship. So in 1710, it passed the world’s first modern copyright law, the Statute of Anne, transferring monopoly rights from the guilds to authors themselves.

A few generations on, the drafters of the Constitution recognized the link between intellectual property protections and innovation. The copyright clause in Article I, Section 8 is the sole enumerated power that identifies a specific, built-in justification: “To promote the Progress of Science and useful Arts”. Over the course of the next century, American courts stretched IP regimes to cover new forms of media like photography, while legislators passed the first modern trademark laws to distinguish high-quality brands.

Even modern technologies have forced institutional rebirth. Consider the United States over the past 150 years. Following its invention in 1874, barbed wire was often called “the devil’s rope” for accelerating the end of cattle drives, the closing of the American frontier, and the expansion of property rights in the West. Samuel Warren was so annoyed by yellow journalism and the use of Kodak cameras at social functions that he and Louis Brandeis wrote a seminal 1890 essay called “The Right to Privacy”, providing a foundation for modern privacy law. Modern antitrust laws and labor protections like workers’ compensation schemes were largely a response to Gilded Age excesses and new classes of injury inflicted during the Second Industrial Revolution. When a North Carolina farmer sued the government in 1946 because military planes were scaring his chickens to death, the Supreme Court ruled that the age-old ad coelum doctrine — “Whoever owns the soil, it is theirs up to heaven and down to hell” — had “no place in the modern world.” In the postwar era, exposés like Unsafe at Any Speed and Silent Spring spawned new consumer welfare protections like mandatory seatbelt laws and pesticide bans. And when the internet unleashed an astonishing array of digital assets and dubious peer-to-peer tools like Napster to share them, courts had to balance protections for creators and preservation of the digital public square.

In each of those cases, technology shaped the law at least as much as the law shaped technology. As Neil Postman wrote:

Introduce the alphabet to a culture and you change its cognitive habits, its social relations, its notions of community, history and religion. Introduce the printing press with movable type, and you do the same. Introduce speed-of-light transmission of images and you make a cultural revolution. Without a vote. Without polemics. Without guerrilla resistance.

Prediction’s Tough (and I’ll Give It a Crack Anyway)

Now we’re caught up, and the obvious question is this: How might AI strain legal systems in the future?

It’s hard to accurately predict the costs and benefits of a new technology, and harder still to forecast what that technology might do to downstream regulatory and judicial institutions. When the Department of Defense introduced GPS in 1973, no one could have foreseen Uber, or designed a robust legal regime for taxicab displacement and the gig economy. Nor is it always clear in the moment what’s happening to the legal field: It wasn’t until the late 19th century that law professors retroactively delineated substantive doctrines like contracts, torts, and property.

If I were working at the Department of Justice or in the judiciary today, I’d therefore stress humility. The law may be sedimentary, but it’s also sticky. As a general rule, we should let legal systems evolve when they must, how they must.

Since I’m not in government, though, I can speculate to my heart’s content about what comes next. AI does seem poised to disrupt a number of foundational legal doctrines:

• What’s “reasonable” in the context of tort law? If diagnostic equipment is more accurate than a human doctor, is it negligent for that doctor to overrule a model?

• Relatedly, what’s foreseeable? To what extent can human operators be accountable for the tortious consequences of black boxes?

• Is an AI model a product, triggering strict liability? A service and information platform, justifying a negligence standard and libel laws? Or something else entirely? (Section 230, which limits online platforms’ liability for what users post and allows for the internet as we now know it, likely doesn’t apply to model developers.)

• How will agents change the “guilty mind” requirement and intent in criminal law? If a misaligned model synthesizes a biological weapon, where does liability fall between developer, distributor, and user?

• How can a contract negotiation result in a meeting of the minds if one or both parties rely on agents? When and how can an agent legally bind a user?

• Copyright law traditionally protects expression of an idea rather than the idea itself, and relies on substantial similarity when testing for infringement. Can either principle survive in the AI era, when expression is easy and different prompts generate similar results?

• Given how ubiquitous and powerful models might be, how should courts contemplate damages? Models are already ubiquitous, and the potential threat to innovation is vast. Could alternative dispute resolution schemes like no-fault compensation be possible in some sectors?

• What will become of state-backed licensing regimes when anyone can use a model to self-diagnose or present a competent legal defense?

• How will evidentiary standards in criminal trials evolve as deepfakes become more convincing? “Pictorial testimony” often requires a human witness to express knowledge of authenticity, but deepfakes may increasingly contaminate memory.

I’m no doubt missing others.

I want to write about some of those topics in the future, and to explore the related question of what emerging technologies will do to nation-states. For now, though, I’ll just say this: Robust legal regimes don’t spring from an expert’s temple fully formed. They’re necessarily iterative, and necessarily susceptible to technological disruption. In that sense, the law is not just a hammer; it’s also a nail.

On a Saturday in June, I drift through the Royal Observatory in Greenwich. I was last here as a boy — no more than 10. It’s not as big as I remember. Few things are.

A lot’s happened at the Observatory over the centuries. Generations of astronomers mapped the stars. Scientists evaluated John Harrison’s H4, the timepiece that revolutionized maritime navigation. (As I browse the exhibit, I wince at the thought of transatlantic trade in the age of dead reckoning.) And the Prime Meridian as we now know it came into being. Historically significant places are spiritual for me, usually. Maybe physical proximity wears thin the veil of time; maybe I just lost my mind in a history class years ago.

But today, for some reason, I can’t find that old reverence. Is it the flukiness of this place? Despite the Observatory’s cultural weight, there’s nothing particularly remarkable about its architecture or location.1 The scientists who worked here could have mounted their telescopes on any secluded hill outside of London. Alternatively, delegates at the 1884 International Meridian Conference could have picked any of the dozens of prime meridians in use at the time for their new global standard. Greenwich won out mainly due to the outsize influence of the Royal Navy.2

No, that’s not it. Greenwich Mean Time is arbitrary, but so are many social constructs. Today, I realize, the detachment I feel’s on my end.

I exit the Observatory; the sun seems to stick to me here. Ducking under a chestnut tree, my thoughts go to time. Specifically, how I’ve spent my own.

My twenties have been wilderness years. I’ve lived in five countries, documented war crimes, monitored a democratic election, conducted global development research, and worked in international advocacy and tech policy. I’ve also had the more dubious privilege of getting detained at border crossings, attending a cult meeting, getting polygraphed, being robbed of $800, recovering the aforementioned $800, coming within spitting distance of a grizzly bear, getting prank-called by Mandy Patinkin, hunkering down in bomb shelters, going to prison (to represent a client), and getting trapped in a four-hour megachurch service. I am a part of all that I have met.

But what of it? Lately, I’ve been feeling… scattered. Like loose change on linoleum. The swashbuckling may be good education, but it’s not the dedication of a lifetime.

I need a framework, I decide. Something to grasp at. In the shade of that chestnut tree, I take down three questions that move me:

1. What might the end of history actually look like (if there is one)?

2. What will emerging technology do to social structures (and vice versa)?

3. What should we do in the meantime?

I pause for a beat. Maybe I should puzzle through those questions with an audience.

I’ve always trod lightly online, whether due to the risk aversion drilled by a family of lawyers, or the innate shame known to all with Irish blood. In the final analysis, though, I see little honor in passive consumption with nothing to show for it. And as a better author once said, I write to find out what I think. If others might get something out of the exercise, so much the better.

Okay. Let’s go.

I hope to post every week or two. Technology, global development, history, social institutions, governance structures, politics, futurism, personal reflections, and pretentious quotes from dead poets are all on the agenda. When I do publish, I’m eager to hear from you — writing can be fairly solitary otherwise.

As for the name: In math and physics, a “random walk” is a series of unpredictable steps. Only when you zoom out does the sequence occasionally reveal larger, more profound patterns. I think of both this project and our world as intellectual random walks. That, and I happen to be a Walker myself. (I originally considered “Drunkard’s Walk” as a title, but decided that’d send a somewhat different message.)

Talk soon.