Anthropic's Olah says AI must be guided from outside Big Tech
Anthropic Co-Founder Calls for Broader Oversight of Artificial Intelligence
By Giselda Vagnoni and Joshua McElwee
AI Oversight Beyond Technology Companies
VATICAN CITY, May 25 (Reuters) - The co-founder of AI company Anthropic said on Monday that the development of artificial intelligence cannot be left solely to technology companies, urging greater oversight from religious leaders, governments and civil society.
Speaking at the presentation of Pope Leo's first encyclical, addressing the challenges posed by artificial intelligence, Chris Olah said there was "a real possibility" that AI will displace human labour "at very large scale".
Moral Imperatives and Societal Impact
"If that happens, supporting those displaced will be a moral imperative of historic proportions," the Canadian said, sitting alongside the pope.
He added that companies like his faced strong commercial, geopolitical and personal pressures that can be at odds with the broader interests of society.
Conflicting Incentives in AI Development
"Every frontier AI lab ... operates inside a set of incentives and constraints that can sometimes conflict with doing the right thing," he said, adding that even well-intentioned researchers remain influenced by those forces.
Olah said this made outside scrutiny essential.
The event marked an unusual convergence between the technology sector and the Catholic Church, which has sought to position itself as a moral voice on the implications of rapid advances in AI.
Anthropic at the Vatican: A Unique Representation
SOLE BIG TECH REPRESENTATIVE AT THE VATICAN
Anthropic is a U.S.-based company that produces the Claude AI tools. It was founded in 2021 by Olah and other former employees of OpenAI, the AI developer created by Sam Altman and Elon Musk, among others, that is behind ChatGPT.
The Anthropic founders left their now-rival over worries that OpenAI was moving too fast without thorough testing.
Ethical Stance and Political Clashes
Anthropic has clashed with President Donald Trump's administration, notably by insisting on guardrails restricting how its models can be used for military purposes such as targeting weapons autonomously or domestic surveillance.
Asked by Reuters why he was the only representative from Big Tech invited to the Vatican event, Olah noted his longstanding focus on AI safety and engagement with religious communities.
"Ultimately, it's the Vatican's decision who they invite," he said, adding he had spent his career working on making AI systems safer and had engaged with more than 15 religions on questions raised by the technology.
Ethical Questions and Global Concerns
'A SCARY MOMENT'
During his speech, Olah welcomed the Church's engagement with the rapidly developing technology, saying the ethical questions raised by AI extended far beyond engineering.
Olah said public concern about AI, particularly among young people, was understandable given the speed of its development.
Risks and Responsibilities
"I think this is a scary moment. Things are moving fast. It's a really powerful technology," he told Reuters.
"There's a risk that things could go badly, and it's incumbent on all of us to push this in a good direction."
Key Areas for Urgent Attention
Olah highlighted three areas he said required urgent attention -- the risk of widespread job losses, the need to ensure that AI benefits are extended worldwide, and the unresolved question of how to interpret increasingly complex and sometimes opaque system behaviour.
Global Distribution of AI Benefits
"AI development is concentrated in a handful of wealthy nations. How can we ensure the gains of AI are shared globally?" Olah said to the audience at the Vatican.
(Reporting by Giselda Vagnoni and Joshua McElwee; Editing by Crispian Balmer and Keith Weir)
