OpenAI putting ‘shiny products’ above safety, says departing researcher | Artificial intelligence (AI)

[Pelican Press 0]

OpenAI putting ‘shiny products’ above safety, says departing researcher | Artificial intelligence (AI)

A former senior employee at OpenAI has said the company behind ChatGPT is prioritising “shiny products” over safety, revealing that he quit after a disagreement over key aims reached “breaking point”.

Jan Leike was a key safety researcher at OpenAI as its co-head of superalignment, ensuring that powerful artificial intelligence systems adhered to human values and aims. His intervention comes before a global artificial intelligence summit in Seoul next week, where politicians, experts and tech executives will discuss oversight of the technology.

Leike resigned days after the San Francisco-based company launched its latest AI model, GPT-4o. His departure means two senior safety figures at OpenAI have left this week following the resignation of Ilya Sutskever, OpenAI’s co-founder and fellow co-head of superalignment.

Leike detailed the reasons for his departure in a thread on X posted on Friday, in which he said safety culture had become a lower priority.

“Over the past years, safety culture and processes have taken a ********* to shiny products,” he wrote.

Yesterday was my last day as head of alignment, superalignment lead, and executive

This is the hidden content, please

• Sign In
• or
• Sign Up

.

— Jan Leike (@janleike)

This is the hidden content, please

• Sign In
• or
• Sign Up

OpenAI was

This is the hidden content, please

• Sign In
• or
• Sign Up

of ensuring that artificial general intelligence, which it describes as “AI systems that are generally smarter than humans”, benefits all of humanity. In his X posts, Leike said he had been disagreeing with OpenAI’s leadership about the company’s priorities for some time but that standoff had “finally reached a breaking point”.

Leike said OpenAI, which has also developed the Dall-E image generator and the Sora video generator, should be investing more resources on issues such as safety, social impact, confidentiality and security for its next generation of models.

“These problems are quite hard to get right, and I am concerned we aren’t on a trajectory to get there,” he wrote, adding that it was getting “******* and *******” for his team to do its research.

“Building smarter-than-human machines is an inherently dangerous endeavour. OpenAI is shouldering an enormous responsibility on behalf of all of humanity,” Leike wrote, adding that OpenAI “must become a safety-first AGI company”.

Sam Altman, OpenAI’s chief executive, responded to Leike’s thread with a post on X thanking his former colleague for his contributions to the company’s safety culture.

skip past newsletter promotion

Alex Hern’s weekly ***** in to how technology is shaping our lives

Privacy Notice: Newsletters may contain info about charities, online ads, and content funded by outside parties. For more information see our Privacy Policy. We use

This is the hidden content, please

• Sign In
• or
• Sign Up

reCaptcha to protect our website and the

This is the hidden content, please

• Sign In
• or
• Sign Up

This is the hidden content, please

• Sign In
• or
• Sign Up

and

This is the hidden content, please

• Sign In
• or
• Sign Up

apply.

after newsletter promotion

“He’s right we have a lot more to do; we are committed to doing it,” he wrote.

Sutskever, who was also OpenAI’s chief scientist, wrote in his X post

This is the hidden content, please

• Sign In
• or
• Sign Up

that he was confident OpenAI “will build AGI that is both safe and beneficial” under its current leadership. Sutskever had initially supported the removal of Altman as OpenAI’s boss last November, before backing his reinstatement after days of internal tumult at the company.

Leike’s warning came as a panel of international AI experts released an

This is the hidden content, please

• Sign In
• or
• Sign Up

on AI safety, which said there was disagreement over the likelihood of powerful AI systems evading human control. However, it warned that regulators could be left trailing by rapid advances in the technology, warning of the “potential disparity between the pace of technological progress and the pace of a regulatory response”.

This is the hidden content, please

• Sign In
• or
• Sign Up

#OpenAI #putting #shiny #products #safety #departing #researcher #Artificial #intelligence

This is the hidden content, please

• Sign In
• or
• Sign Up