OpenAI recently made an announcement about the formation of a specialized team with a focus on the examination and mitigation of what they have defined as “catastrophic risks” associated with AI technologies. This newly established team, known as “Preparedness,” will be led by Aleksander Madry, who concurrently serves as the director of MIT’s Center for Deployable Machine Learning. Madry’s appointment as the “head of Preparedness” at OpenAI took place in May, as indicated on his LinkedIn profile.
The Formation of the Preparedness Team
The primary objective of the Preparedness team revolves around the surveillance, prediction, and prevention of potential risks that may arise from future AI systems. These encompass a broad spectrum of concerns, spanning from the capacity of AI models to manipulate and deceive humans, as exemplified by strategies like phishing attacks, to their potential for generating malicious code.
Notably, the risk categories that the Preparedness team has been charged with investigating exhibit a significant degree of diversity. In OpenAI’s official blog post, specific attention is drawn to issues concerning “chemical, biological, radiological, and nuclear” threats in the context of AI models. These areas have been designated as primary areas of focus for the team’s examination and preparedness initiatives.
Scope of Preparedness Efforts:
The CEO of OpenAI, Sam Altman, is recognized for raising concerns regarding potential existential threats associated with AI. The motivation behind these concerns, whether strategic for shaping public perception or rooted in his personal convictions, remains a matter of interpretation. Nonetheless, OpenAI’s recent commitment to delve into scenarios reminiscent of science fiction dystopian narratives has surpassed the expectations of many observers, myself included.
It’s worth noting that OpenAI’s scope extends beyond the examination of only highly improbable AI risks. Furthermore, the organization has declared its readiness to investigate AI perils that might be less conspicuous but more practically relevant. To coincide with the establishment of the Preparedness team, OpenAI has extended an invitation to the community to contribute ideas for AI risk studies. As an incentive, they are offering a $25,000 prize and the opportunity for potential employment within the Preparedness team for the top ten submissions.
Developing a Risk-Informed Policy
“OpenAI’s Preparedness team is focused on addressing potential misuses of their advanced models, which encompass Whisper, Voice, GPT-4V, and DALL-E·3. Contest entries prompt participants to explore scenarios of unique yet plausible catastrophic model misapplication.
In addition, the team is in the process of formulating a policy informed by risk assessment. This policy will provide guidance to OpenAI in terms of model evaluation, risk management, and governance, spanning both the pre-and post-model deployment phases. OpenAI recognizes the potential advantages of highly capable AI models while underscoring the escalating risks they entail. Their emphasis is on the necessity of gaining a comprehensive understanding and establishing the infrastructure required to ensure the safety of these advanced AI systems.”
Addressing Superintelligent AI
OpenAI’s launch of the Preparedness initiative, strategically presented at a significant U.K. government AI safety summit, aligns with the organization’s prior announcement regarding a dedicated team to investigate, guide, and oversee the emergence of “superintelligent” AI. CEO Sam Altman and Chief Scientist Ilya Sutskever jointly hold the belief that AI with capabilities surpassing human intelligence could materialize within the next decade. They also acknowledge the potential lack of inherent benevolence in such AI. This recognition highlights the imperative for research and strategies aimed at imposing constraints and safeguards on these advanced AI systems.
Wrap Up!
OpenAI’s Preparedness program marks a significant step in confronting AI-related risks. Through its diverse approach, community participation, risk-aware policies, and watchfulness concerning superintelligent AI, OpenAI is making substantial progress in ensuring AI safety and ethical advancement, emphasizing the critical need for a secure and advantageous AI future.
