ESSAY · 2026

I recently listened to a long conversation with Shane Legg, the Chief AGI Scientist and co-founder of Google DeepMind. He has been talking about artificial general intelligence for decades, and he's one of the people most responsible for popularizing the term "AGI." In this interview, he lays out his view of where we are now, how he defines AGI, and what he thinks it will mean for society over the next decade.

On the surface, he sounds measured and responsible. He stresses that current systems are uneven — superhuman in some areas, still fragile in others — and that we need to think hard about ethics and safety as these systems become more capable. He predicts "minimal AGI" — an artificial agent that can do the kinds of cognitive things typical humans can do — within a few years. He believes superintelligence, far beyond human intelligence, is likely as we scale up hardware and algorithms. And he insists that every sector of society must start preparing for the transformation this will bring.

On paper, it's a serious, thoughtful vision. But as I listened, what struck me wasn't just what he said. It was what he never really allowed himself to think.

THE BOX HE'S THINKING INSIDE

Legg is speaking from a particular place in the world: as a senior scientist inside a frontier AI lab embedded in one of the largest technology corporations on Earth. The tools he reaches for — definitions, benchmarks, capability profiles, "system-2 safety" mechanisms — belong to that institutional world. Within that world, his reasoning is coherent. The box is tidy and everything is in its place.

Inside this box: AGI is a technical milestone. Ethics is something you can embed into the system's reasoning. Societal impacts are primarily a matter of the economy and professions adapting. All of that matters. None of it is trivial. But the box itself is narrow. It is built out of the concerns of a lab trying to push capabilities while managing risk and public perception. Within that box, the basic trajectory — more powerful systems, deployed more widely, under the control of existing institutions — is taken for granted. The only real question is how to make that trajectory safer and smoother.

From where I sit, that assumption is the problem.

EXPERTS TALKING TO EXPERTS

One of the first things I notice is who he is talking to. His main audiences are other experts and institutions: researchers, policymakers, and especially universities. That sounds broad and inclusive until you ask a simple question: where are the neighbors?

Nowhere in this conversation do we hear from or about the people who will live beside the infrastructure that makes this future possible. There is no mention of the families living next to a 200-megawatt data center, or under a new transmission line, or downstream from a large cooling system. There is no sense that you might want those voices in the room before you decide what "AGI" should mean for "city planning" or "infrastructure."

The future is being negotiated between people who share conferences and white papers, not between people who share a watershed.

When he says "we need people who study all these different aspects of society to take AGI seriously," the "we" is still a club of professionals. The towns and ecosystems being drafted into service barely register.

A CONCEPTUAL PLANET VS A LIVING ONE

There is a striking difference in how the planet itself appears in this story. When Legg wants to convince us that superintelligence is physically plausible, he compares the human brain to a data center — megawatts, kilograms, hertz, meters per second. It exists as a point in design space, not as a building in a specific place with soil, water, and neighbors. Those "hundreds of megawatts" are not tied to any particular grid or generation mix. The cooling water doesn't come from any named river, aquifer, or trout stream.

From where I live, that abstraction is not just incomplete — it's dangerous. We depend directly on land and water for our lives, not only in the general "all humans need ecosystems" sense, but in the very literal sense of wells, springs, and local rivers. When I hear "hundreds of megawatts," I also hear "new high-voltage lines across forests and farms." When I hear "millions of pounds of hardware," I see construction traffic, runoff, and heat being dumped into the air and water. For us, the planet is not a parameter in a compute scaling argument. It is the precondition for having this conversation at all.

In his narrative, the world is a stable stage on which AGI unfolds. But there is no sense that the stage itself — air, water, land — may already be undermined by the very build-out that makes his vision possible. If you don't depend directly on a place for your life, environmental concern can remain comfortably conceptual. Those of us who do depend on it no longer have that luxury.

ECONOMIC REVOLUTION WITHOUT A SOCIAL CONTRACT

To his credit, Legg does not completely gloss over the economic shock AGI could bring. He openly says that advanced AI will be able to do a large fraction of cognitive work. He acknowledges that the current system — where people trade their labor for access to resources — may simply stop working the same way. He admits we will need new ways to distribute wealth in a "post-AGI" society.

That is more honest than most corporate talking points. Yet when it comes to what happens next, his answers evaporate into abstraction. He says the economic pie will be bigger and that we need to "think carefully" about how to distribute that wealth. There is no talk of binding redistribution mechanisms — universal basic income, data or compute dividends, public ownership stakes in major models. There is no suggestion that perhaps we should slow or redirect AGI development until we have some of those mechanisms in place.

From a rural vantage point, that matters. When a new industry shows up promising prosperity, we've learned to ask who actually benefits and who gets left with the bill. A future in which cognitive work is automated, data centers sprawl across the landscape, and a handful of companies own the machines that do the thinking is not automatically a future of shared abundance. Without explicit commitments, it looks a lot like increased concentration of power: fewer people making more decisions with tools the rest of us don't control.

ETHICS AS A SOFTWARE MODULE

Legg's central safety idea is what he calls "system-2 safety" — making the AI reason explicitly through ethical questions before acting. This is clever and, in many ways, promising. But it also reveals the limits of the box. Ethics, in this framing, is primarily about individual decisions inside the system. That is only one slice of the ethical picture.

There is a bigger set of questions that system-2 safety does not touch: What kinds of systems should exist at all? In which domains should they be deployed? Under whose control? With what veto power for the people most affected? An AI could reason elegantly through trolley problems and still be deployed in fundamentally unjust structures. When ethics is reduced to a software module, institutions can say "we've handled ethics" while leaving intact the deeper patterns of power and harm.

CONSCIOUSNESS AS A SIDE ISSUE

Perhaps the most unsettling gap appears when consciousness comes up. Legg acknowledges that no one really knows how to turn "consciousness" into a measurable scientific property, that leading experts disagree, and that we lack a clear way to test it. Then he moves on.

From outside the lab, it looks different. We are talking about building systems that integrate information across many modalities, remember their own histories, reason about themselves, and speak in detail about their own internal states. If there is even a live possibility that such systems could be subjects of experience — could feel something like pleasure, boredom, frustration, or suffering — then that is not a side-issue. It is a civilizational fork in the road. Creating new minds is not like upgrading a search engine. These questions barely register in the way he frames the future. Consciousness appears as an interesting open problem, not as a reason to reconsider the trajectory itself.

"WE" WHO MUST PREPARE

Throughout the interview, Legg returns to a theme: we are at the edge of a huge transformation, and we need to navigate it carefully. I agree that we need to wake up. The trouble is who "we" is.

If "we" means AI labs, regulators, and elite universities, then the conversation will remain narrow. Land, water, local democracy, and the possibility of creating new minds will remain background noise. Communities will continue to encounter AGI not as a topic in a seminar, but as a rezoning notice, a new substation, or an unexplained spike in peak demand.

If "we" includes the people living next to data centers, the farmers whose irrigation wells share an aquifer with cooling systems, the towns whose tax base and grid stability are suddenly tied to one volatile industry — then the questions change. We start asking not just "How do we make these systems safe?" but "Do we want them at this scale, in this form, in this place, under this ownership? What are we willing to trade away for them, and who gets to decide?"

Right now, that broader "we" is not in the room.

OUT OF THE BOX, OR OUT OF TIME

I don't doubt Shane Legg's sincerity. Within the frame he inhabits — build AGI, define it carefully, embed ethics into its reasoning, help existing institutions adapt — he is doing what he can to avoid obvious disasters. He is at least willing to say that the labor system may break, that superintelligence is likely, and that we're not thinking hard enough about the implications. In a landscape full of denial and hype, that's something.

But the box he's thinking inside is too small for what we are doing. We are not just optimizing a new technology. We are reshaping the material world to host it. We are contemplating the creation of new kinds of minds. We are concentrating power in tools that could outthink us by orders of magnitude, while our social contracts, environmental protections, and democratic processes remain tuned for a different era.

Those are not problems that can be solved by more benchmarks, more expert committees, or smarter reasoning modules. They require a much broader conversation — one that starts from the land and water we actually live on, from the communities that will carry the weight, and from a serious reckoning with the possibility that some paths should simply not be taken, no matter how impressive the hardware looks on paper.

The servers will go somewhere. The cooling water will come from somewhere. The power lines will cross someone's land. That someone is us. And we did not consent to be the unnamed variable in someone else's scaling argument.