Anthropic’s Chris Olah warns the Vatican about frontier AI incentives

Pope Leo XIV released Magnifica humanitas, an encyclical focused on safeguarding the human person in the age of artificial intelligence. At its Vatican City presentation, Anthropic co-founder Chris Olah moved the discussion beyond general promises toward the forces that actually shape development of the most advanced models.

Frontier AI development is shaped by incentives that can conflict with the public good

Olah warned that frontier AI labs do not operate in a neutral environment. They are pulled by commercial interests, research competition, geopolitics, pride and ambition. These incentives can support speed and innovation, but they can also conflict with what is right for society.

Olah also drew a line between classical engineering and today's AI. A bridge is designed from parts with clear functions. Large models are more like systems grown from vast amounts of human words. Their capabilities emerge during training, and some of their internal states remain hard to read even for their builders.

Trusting labs is not enough: governance needs voices from outside

For Olah, governance cannot remain inside the companies alone. External critics, faith communities, civil society, scholars and governments are needed. Each group brings a different form of accountability: moral language, lived social experience, independent expertise or democratic authority.

The appearance of a researcher from one of the largest AI labs at a Vatican presentation itself shows how the AI safety debate is expanding beyond technical conferences and regulatory bodies. It is drawing in institutions that have shaped humanity's ethical frameworks for centuries.

The diagnosis is strong, concrete leverage is still missing

Olah's remarks are strong as a diagnosis, weaker as a roadmap. The call for external pressure, faith communities and civil society is right, but it does not specify what labs must do differently. Meaningful institutional pressure on AI labs currently comes more from regulators and investors than from moral institutions.

The question remains whether institutions like the Vatican hold leverage or only a voice. In debates over laws, standards and licenses for frontier models, technologists and lawyers still dominate.

Three questions that connect the technical with moral responsibility

The remarks named three practical questions. The first concerns the global poor, labor displacement and whether AI gains will be shared globally. The second asks what human flourishing means beyond productivity. The third calls for discernment about the nature of AI models and their mysterious internal states. This is where technical debate meets moral and political responsibility.

Lilith's verdict

The strongest part of Olah's intervention is institutional rather than technical: even sincere researchers work inside a race shaped by commerce, prestige and geopolitics. If AI is to serve people, trusting labs is not enough. It needs sustained outside pressure, a language of human dignity and the courage to ask who carries the costs and who receives the gains.