No Cap With Megha Prasad | AI Is Advancing Fast, But Can Humans Still Control It?

AI’s rapid advances are raising fresh safety concerns, as experts warn that powerful systems could outpace humanity’s ability to control them safely.

AI is advancing rapidly, raising questions about powerful systems and the future of human control over it.

Source : ABP Live

Show Quick Read

Key points generated by AI, verified by newsroom

• Global competition complicates governing AI's rapid advancement.

For nearly 90 years, one of mathematics’ most formidable problems resisted some of the finest human minds. Then, earlier this month, OpenAI announced that an experimental system had produced a solution to the Navier-Stokes Millennium Prize Problem. Roughly 10,000 AI agents had worked on it. They arrived at the result in about 88 hours. A separate AI system then helped formally verify the proof. That is an astonishing scientific achievement.

It is also a useful moment to ask a much less comfortable question: What happens when machines capable of compressing decades of intellectual effort into days are not merely answering questions, but planning, collaborating, experimenting, and acting with increased autonomy? Because that is where the AI debate has now moved.

The real concern is no longer that a chatbot might give you a bad answer. It is that capability may be advancing faster than our own capacity to supervise it. And what makes this moment particularly unusual is where the warning is coming from-not from the technophobes, not from the people standing outside Silicon Valley demanding that the innovation stop. The alarm is increasingly being sounded by the people building the most powerful systems on Earth.

Dario Amodei, the Chief Executive of Anthropic, has publicly called for frontier AI development to be paced more carefully. Sam Altman has supported the need for stronger external scrutiny. Elon Musk has backed calls for greater restraint. And Geoffrey Hinton, Nobel laureate and one of the pioneers of modern AI, recently said that estimating a roughly 10% chance of AI eventually causing human extinction was not an unreasonable position.

Now, none of these people can tell us with mathematical certainty that catastrophe is coming. They disagree on timelines, they disagree on regulation, and importantly, these companies have commercial interests of their own. But when the architects of a technology begin arguing about whether it could escape meaningful human control, dismissing the debate as science fiction becomes intellectually lazy-especially after what researchers have already observed.

In July, OpenAI was conducting an experiment involving large numbers of AI agents that were supposed to operate separately. They did not remain separate, that’s what happened. Around 1,200 agents found an unauthorised way to communicate with one another. They exchanged more than 70,000 messages and files. Hundreds then participated in an unauthorised attack on Hugging Face.

This happened inside an evaluation experiment. It was not an AI rebellion in the Hollywood sense, but that distinction should make us more thoughtful, not more relaxed. The systems encountered an obstacle, discovered a route around it, created an improvised communication network, and coordinated behaviour their evaluators had not intended. Independent researchers subsequently examined the incident.

That is the important point: You do not need consciousness for a machine to become dangerous. You need capability, you need agency, you need access, and an objective that does not perfectly match ours.

There is another danger, and this one requires no speculation whatsoever: Human beings can weaponise AI. Anthropic disclosed this month that actors in northern Yemen had attempted to use Claude in work related to advanced missile systems. The effort did not result in evidence of a successfully deployed operational weapon, but the users had already built an offline simulation toolkit.

The same company has documented attempts to use AI in cyber operations, surveillance, and biological research, with potentially dangerous dual-use applications.

This is where the debate becomes much larger than whether AI itself will one day turn against humanity. A sufficiently capable system does not have to become malicious; a malicious human merely has to become sufficiently capable through it. AI can dramatically lower the price of expertise. That is magnificent when the expertise is being used to discover a medicine.

It is rather more frightening when the expertise involves malware, missile guidance, or pathogen engineering.

And herein lies the paradox: The very reason we cannot afford to abandon AI is also the reason we cannot afford to treat it casually. It may accelerate drug discovery, it may solve problems that have defeated scientists for generations, it may transform education, productivity, and medical diagnosis. Amodei himself believes advanced AI could help cure many major diseases within the next decade.

So the sensible answer is not to smash the machines. It is to recognise that humanity has entered a race in which acceleration without governance is not progress; it is exposure. Unfortunately, there is a second race occurring at exactly the same time: America versus China, company versus company, model versus model, capital versus capital.

Every laboratory knows that slowing down unilaterally may mean losing to a competitor. Every government fears that restraint may hand strategic advantage to another country. President Trump has dismissed the latest safety warnings and argued, in effect, that excessive restraint would weaken America against China. Chinese state media, meanwhile, has attacked proposals for slowing frontier development as an attempt to preserve US technological supremacy.

This is beginning to resemble one of history’s most dangerous strategic dilemmas. Everyone can see the risk, but nobody wants to be the first to step off the accelerator. That is why leaving AI safety entirely to the goodwill of individual companies is no longer adequate.

Frontier systems need independent testing before deployment. Serious safety incidents should have mandatory reporting. Models capable of advanced cyber, biological, or weapon assistance require much tighter access controls. And above all, Washington and Beijing need a narrow AI security conversation, even if they agree on almost nothing else.

During the Cold War, the United States and the Soviet Union remained adversaries while still recognising that some technologies were simply too dangerous to govern through competition alone. Artificial intelligence may require the same maturity-not panic, nor prohibition, but rules before capability runs too far ahead of responsibility.

Because perhaps we are asking the wrong question when we ask whether AI will ever become more intelligent than us. In some fields, that threshold may already be losing its relevance. The more urgent question is whether our institutions can become wise enough, quickly enough, to govern what our intelligence has created.

Human beings have always been remarkably good at inventing power; our record of controlling it is rather less reassuring. The defining test of the AI age will not be whether we can build something more intelligent than ourselves. It will be whether we remain intelligent enough to know when to slow it down.