The development and deployment of Artificial Intelligence at various levels has helped solve many complex problems quickly and accurately. The computational power of machine learning based on AI is getting stronger and stronger day by day, which has resulted in a debate on its regulations. Though many AI regulations have been framed and passed recently, no mechanism has been developed that can provide 100% surety and security to address its misuse and manipulation. AI in 2024: Top Predictions and Trends to Watch
To address the global challenge of AI supervision, OpenAI is among the most popular AI research and development organisations developing a ‘Superhuman AI’. According to the OpeaAI on superintelligence, AI is vastly smarter than humans and could be developed within the next ten years. However, we still do not know how to reliably steer and control superhuman AI systems. Solving this problem is essential for ensuring that even the most advanced AI systems in the future remain safe and beneficial to humanity. Must Read: OpenAI Initiates GPT-5 Development, Aiming for Superintelligence
Superintelligence can have both positive and negative impacts on humans and society. Superintelligence will be the most impactful technology humanity has ever invented and could help us solve many of the world’s most important problems. But the vast power of superintelligence could also be very dangerous and could lead to the disempowerment of humanity or even human extinction, says the OpenAI report on Introducing Superalignment.
Why OpenAI is Developing Superhuman Intellegence AI system?
A core challenge for aligning future superhuman AI systems (superalignment) is that humans will need to supervise AI systems much smarter than them. The research studies a simple analogy of how small models can supervise large models. The research studies show that the use of a GPT-2-level model to elicit most of GPT-4’s capabilities—close to GPT-3.5-level performance—generalizes correctly even to hard problems where the small model failed. This opens up a new research direction that allows us to directly tackle a central challenge of aligning future superhuman models while making iterative empirical progress today.
- OpenAI formed the Superalignment team in July to develop ways to steer, regulate, and govern “superintelligent” AI systems.
- Some experts say there’s little evidence to suggest that the startup’s technology will gain world-ending, human-outsmarting capabilities anytime soon.
- OpenAI is launching a $10 million grant program to support technical research on superintelligent alignment.
- A portion of funding for the grant will come from former Google CEO and chairman Eric Schmidt.
- Schmidt is fast becoming a poster child for AI doomers, asserting the arrival of dangerous AI systems is nigh.
Must Do: 10 Must-Watch Artificial Intelligence Movies That Will Blow Your Mind!
OpenAI’s goal is to build a roughly human-level automated alignment researcher where they can use vast amounts of computing to scale our efforts and iteratively align superintelligence. To be the first automated alignment researcher, OpenAI needs to develop a scalable training method, followed by validation of the resulting model, and finally stress test the entire alignment pipeline.
OpenAI’s superhuman AI systems will provide a training signal on tasks that are difficult for humans to evaluate and can leverage AI systems to assist in the evaluation of other AI systems (scalable oversight). In addition, the organisation wants to understand and control how models generalize our oversight to tasks humans can’t supervise (generalization). To validate the alignment of AI systems, OpenAI automates the search for problematic behaviour (robustness) and problematic internals (automated interpretability). Finally, superhuman AI intelligence can test the entire pipeline by deliberately training misaligned models and confirming that our techniques detect the worst kinds of misalignments (adversarial testing).
According to the report, current alignment methods, such as reinforcement learning from human feedback (RLHF), rely on human supervision. However, future AI systems will be capable of extremely complex and creative behaviors that will make it hard for humans to reliably supervise them. For example, superhuman models may be able to write millions of lines of novel—and potentially dangerous—computer code that would be very hard even for expert humans to understand.
Relative to superhuman AI models, humans will be “weak supervisors.” This is a core challenge for AGI alignment: how can weak supervisors trust and control substantially stronger models? Says the OpenAI report Weak-to-strong generalization
Must Learn: Mistral 7B Tutorial: A Step-by-Step Guide on How to Use Mistral LLM
OpenAI’s Superhuman Intellegence AI: Set Up
To make progress on the core challenge of superhuman intelligence, OpenAI proposes an analogy in an empirical study where a smaller (less capable) model supervises a larger (more capable) model. A simple analogy for superalignment: In traditional machine learning (ML), humans supervise AI systems weaker than themselves (left). To align superintelligence, humans will instead need to supervise AI systems smarter than them (center). We cannot directly study this problem today, but we can study a simple analogy: can small models supervise larger models (right)?
The report also suggests that it is not expected that a strong model will perform better than the weak supervisor that provides its training signal—it may simply learn to imitate all the errors the weak supervisor makes. On the other hand, strong pretrained models have excellent raw capabilities—we don’t need to teach them new tasks from scratch; we just need to elicit their latent knowledge. The critical question is then: will the strong model generalize according to the weak supervisor’s underlying intent—leveraging its full capabilities to solve the task even on difficult problems where the weak supervisor can only provide incomplete or flawed training labels?
Why $10 Million Grant for OpenAI’s Superhuman Intelligence AI Development
The OpenAI research on Superhuman Intelligence shows naive human supervision—such as reinforcement learning from human feedback (RLHF)—could scale poorly to superhuman models without further work, but it is feasible to substantially improve weak-to-strong generalization.
To strengthen the capability of the current empirical setup and the ultimate problem of aligning superhuman models, OpenAI has launched a grant of $10 million. This grant for the AI & ML communities will help in developing future models that can imitate weak human errors rather than current strong models that imitate weak model errors, which could make generalization harder in the future.
We believe this is an exciting opportunity for the ML research community to make progress on alignment. To kickstart more research in this area,
OpenAI has released the open source code to make it easy to get started with weak-to-strong generalization experiments. The organization will also fund a $10 million grant program for graduate students, academics, and other researchers to work on superhuman AI alignment broadly.
