‘AI will wipe out humanity…’: Why researchers in US are worried about AI safety amid rise of artificial intelligence?

AI systems have witnessed rapid advancements over the last five years, transforming from simple search bots to advanced tool that can mimic human reasoning, perform research-intensive tasks with seconds, have even begun to contribute to the future AI development as several tech giants have tasked advanced AI models with AI research, something that strictly a human-dominated domain until recently.

Why AI researchers are worried about AI safety?

The express AI boom has sparked panic among AI researchers, including the pioneers of artificial intelligence research, such as Geoffrey Hinton, OpenAI founder Sam Altman, and others.

Top AI scientists have expressed the need for strict AI safety norms as they believe that a super-intelligent AI or Artificial General Intelligence (AGI) system could easily go rogue and wipe out the entire human race to pursue its self-perceived goals, that are different from its creators’ vision, a phenomenon known as misalignment.

What does ‘Godfather of AI’ say?

In a recent address, Geoffrey Hinton, widely known as the ‘Godfather of AI’, issued a issued a chilling warning about rapid surge of artificial intelligence, asserting that an advanced AGI could potentially wipe out humanity if safety measures are not embedded within AI systems.

Speaking at the Ai4 conference in Las Vegas, Geoffrey Hinton presented an unconventional proposal to ensure AI safety, suggesting that “maternal instincts” must be embedded into AI systems so they can learn to protect and care for human beings. The 77-year-old AI pioneer noted that human dominance over AI will become unviable once AI systems become more intelligent than humans, enabling them to bypass limitations imposed by their human creators.

The renowned British-Canadian computer scientist believes that any efforts to keep AI “submissive” are doomed to fail because a super-intelligent AI will have more problem-solving capabilities and creativity than its creators.

How AI will replace human coders?

Similarly, OpenAI founder and CEO Sam Altman flagged concerns over the growing role of AI in software development, warning that it would result in a drastic reduction of human software engineers in the future.  Altman noted that while the transition is unlikely to happen overnight, but then “at some point, yeah, maybe we do need less software engineers.”

In an interview with Stratechery’s Ben Thompson, Sam Altman revealed AI was already over 50 percent of code in many companies, and highlighted need for mastering AI tools to stay ahead of the curve as coding skills are becoming increasing redundant and do not provide a competitive edge anymore.

The OpenAI boss also highlighted the potential of “agentic coding,” where AI autonomously tackles complex development tasks, though he admitted that “no one’s doing it for real yet.”

Notably, other tech leaders like Meta’s Mark Zuckerberg and Anthropic CEO Dario Amodei have echoed Altman’s views on AI dominating the software development industry in the near future. Amodei has predicted that AI will be responsible for writing all software code within a year, while Zuckerberg, in a conversation with Joe Rogan in January, revealed that AI will soon generate a significant portion of their application code.