Apple co-founder joins top scientists, leaders in call to demand pause on AI superintelligence development

Many prominent figures including Apple co-founder Steve Wozniak and Virgin Group founder Richard Branson have called for an interim ban on the development of AI superintelligence. The list which includes over a 1,000 personalities also includes the names of computer scientists Yoshua Bengio and Geoff Hinton, who are considered the ‘Godfather of AI’.

Other leading figures who are signatory to the letter also include author Yuval Noah Harari, Former U.S. National Security Adviser Susan Rice, actor Stephen Fry and Prince Harry.

The signatories argue that development of superintelligence poses significant risks of human freedom, civil libery, dignity, control and even extinction. They have demanded for interim ban on the development of he technology until it can be developed safely and controllably with strong public buy-in.

“Innovative AI tools may bring unprecedented health and prosperity. However, alongside tools, many leading AI companies have the stated goal of building superintelligence in the coming decade that can significantly outperform all humans on essentially all cognitive tasks. This has raised concerns, ranging from human economic obsolescence and disempowerment, losses of freedom, civil liberties, dignity, and control, to national security risks and even potential human extinction.” the signatories argue

“We call for a prohibition on the development of superintelligence, not lifted before there is broad scientific consensus that it will be done safely and controllably, and strong public buy-in.” they added

What is Superintelligence?

Superintelligence or Artificial General Intelligence is a hypothetical stage in the development of an AI system when the technology surpasses humans in almost all cognitive tasks. However, there is no clear definition of Superintelligence in the field and each tech company approaches the topic with a slighlty different lens.

However, there has been a push from almost all the frontier AI labs for development of Superintelligence. OpenAI CEO Sam Altman predicts the rise of superintelligence by 2030 while Meta CEO Mark Zuckerberg has said that Superintelligence is ‘now in sight’.

Some tech leaders are optimistic about the arrival of new technology while other leaders have worried that its development could go on to lead to even the annihilation of the human race.

In a recent interaction, Bengio had warned about emergence of AI with agency, with the capacity to form and execute its own plans. He noted that such agents could engage in deceptive or manipulative behavior to achieve their goals and could potentially turning against humans.