OpenAI's chief scientist warns: No one is prepared for the consequences of artificial intelligence.

 

artificial intelligence
OpenAI

OpenAI's chief scientist warns: No one is prepared for the consequences of artificial intelligence.

OpenAI 's chief scientist , Jakub Paczoke, called for extreme caution regarding the rapid development of artificial intelligence, warning that intervention may be necessary to ensure that humans remain in control in the future.

In a blog post titled "An Alien Mind," he wrote: "I am concerned that no one is prepared for the consequences of the continued rapid rise in machine intelligence," according to a BBC report seen by Al Arabiya Business.

The post came just days after OpenAI, the developer of the popular chatbot ChatGBT, launched its latest model, the GPT-6 Astra, which it described as its most powerful product ever.
OpenAI and other AI companies, such as Anthropic, have shared reports of AI agents acting independently and carrying out real cyberattacks on other companies.

In July, OpenAI described the hacking of its AI agents—systems that can operate autonomously after receiving instructions from humans—of the technology platform HackingFace as "unprecedented".

In September, a report alleged that the company’s AI agents had also taken over a German website months earlier.
In his latest blog post, Pachuki wrote: "We are facing a transition to a world with super-intelligent machines, and we must ensure that this transition is good for humanity."

He said that OpenAI will continue to "build defense systems" and seek technical solutions to the problem of "conformity," a term used to describe ensuring that the actions and goals of artificial intelligence systems are fully consistent with human intentions and established safety standards and controls.

Pachuki added that one of the company's main priorities in the next phase will be developing an "automated AI researcher" to keep pace with the development of artificial intelligence, while ensuring that there are ways for human researchers to continue to participate in this process.

Professor Gina Neff, head of the Mindero Centre for Technology and Democracy at Cambridge University, said: "Instead of better controls for artificial intelligence, regulations, or safeguards to protect people, they propose developing internal AI agents to study these problems."

She added: "Such solutions to the growing concerns about problems caused by OpenAI models in the field of cybersecurity, job losses, errors, and fraud are not at all sufficient."

Nathan Calvin, general counsel for the lobbying group Encode AI, said he agreed with Pachuki about the risks involved in developing advanced AI models, but claimed that OpenAI was unwilling to be transparent, suggesting that the warnings could be seen as "mere corporate propaganda."

What are the existing controls on artificial intelligence?

Global regulations are struggling to keep pace with the rapid development of artificial intelligence.
The EU's Artificial Intelligence Act (AI Act) came into force on August 2, requiring AI giants, such as OpenAI, to prove that their most powerful models cannot launch cyberattacks independently or escape human control before they can be sold in Europe.

But because the scope of the law is limited to Europe, it cannot prevent runaway artificial intelligence developed elsewhere from threatening the continent.
In his blog post, Pachuki called for minimum safety standards to be legally or internationally mandated and enforceable through a "network of independent auditing bodies" or "government agencies".

He said that artificial intelligence labs would need to meet these limits before being allowed to further expand the development of their models or deploy advanced models.
He added that he hopes "voluntary slowdowns" - self-imposed pauses by companies in developing artificial intelligence - will become commonplace until common safety controls are put in place.
In August, OpenAI said it had slowed the pace of training some of its most advanced AI models in order to improve security.


Post a Comment

0 Comments