OpenAI strengthens safety measures against AI risks

OpenAI is taking proactive steps to enhance its internal safety processes to address potential threats from harmful AI. The company is introducing a ‘safety advisory group’ that will hold authority over technical teams, providing recommendations to its leadership. While the board has been granted veto power, it remains to be seen whether it will exercise this authority.

In response to recent leadership changes and an evolving discourse on AI risks, OpenAI has released an updated ‘Preparedness Framework.’ This framework establishes a protocol for identifying, analysing, and managing ‘catastrophic’ risks associated with the models under development. OpenAI defines catastrophic risks as those that could lead to economic damage or severe harm to individuals, including existential risks.

The framework details risk levels and criteria, ensuring transparency and consistency in the evaluation process. OpenAI is establishing a cross-functional Safety Advisory Group to review technical reports and provide recommendations from a higher vantage point. This group aims to uncover potential ‘unknown unknowns,’ though the inherent challenges of identifying such risks are acknowledged.

Recommendations from the Safety Advisory Group will be sent simultaneously to the board and leadership. Leadership, including CEO Sam Altman and CTO Mira Murati, will make the final decision on whether to proceed with or halt the development of a model, with the board retaining the power to reverse these decisions.