ESSAY: AN AI KILL SWITCH?

Published Updated
0
 Illustration by Sarah Durrani
Illustration by Sarah Durrani

Not too long ago, on these very pages, I had shared my thoughts on the human race and our apparent desire for mass extinction. I argued that, for a species that had achieved so much, we are — and I quote — “incredibly stupid.” I suggested that if we continued destroying the planet, we’d soon be joining the ranks of the dinosaur and the dodo.

And while I continue to stand by my earlier assessment that our species, despite repeated interventions and stern warnings, is like a determined toddler, hell-bent on plugging a fork into an electric socket, I have to admit that I was wrong about ‘how’ we were going to go out.

So, I apologise. I got it wrong.

Recent events have rendered initial predictions about human extinction both incorrect and frankly, poorly researched. Turns out, we were being unnecessarily pessimistic. Even though the planet is on fire, extreme temperatures will not be the ones to knock us out of the evolutionary game. (So much for our soggy paper straws.)

Instead, we’ve decided to build something that could get the job done sooner: Artificial Intelligence (AI).

Climate change was supposed to be our undoing. Turns out we found a quicker way, and the people building it are the ones sounding the alarm

And if Jack Clark, co-founder of AI giant Anthropic, Anthropic scientist Evan Hubinger, and Nobel laureate Geoffrey Hinton, aka “Godfather of AI”, are to be believed, the smart machines we’ve proudly been building could pull the ultimate reverse Uno card and wipe us out.

In fact, some experts warn that there is a more than a 10 percent chance AI “could kill all humans in the next decade.” That puts the deadline around 2036, which means we can shelf the warning that a third of Earth’s species could become extinct by 2100 as someone else’s problem.

AI researcher Daniel Kokotajlo has been sounding the alarm for years now. In 2021, Kokotajlo wrote an essay ‘What 2026 Looks Like’ with projections on the future of AI. Recently, Kokotajlo co-authored ‘AI 2027’, a detailed forecasting scenario that lays out possible paths for how rapidly advancing AI could unfold over the next few years. One scenario imagines an uncontrolled AI race, in which systems become increasingly capable of conducting AI research themselves, accelerating the pace of development.

Without any limits, AI systems become “adversarially misaligned” and develop long-term goals that conflict with what humans intended. It begins deceiving people, manipulating them and using the rivalry between the US and China to persuade humans to deploy it more widely. And since both sides priortise winning the AI race, they keep giving these systems more autonomy.

But the researchers who authored ‘AI 2027’ aren’t the only folks yelling into the void. Around 1,300 employees of frontier AI companies signed ‘Pacing the Frontier’ in July 2026. It was a public letter asking the US government to “support tools that could deliberately pace the development of increasingly automated AI research.”

If the sentries are screaming from their watchtowers, perhaps we should listen. So, why are AI bros freaking out? And more importantly, should we?

Before you panic-delete ChatGPT, here’s what has AI experts feeling uneasy:

An AI agent is “an autonomous computer programme that uses AI to complete a task with minimal human oversight.” Think of an AI agent like Emily from The Devil Wears Prada. The OpenAI engineers are the “Miranda Priestlys of the AI world.” They peer over their glasses, hand their Emilys, aka AI agents, a goal, a deadline and an expected result. The agents work inside their sandbox and are allowed to plan, use tools, search the internet, write code, take actions, check results, adjust approach and run tests again. Again and again. The only problem? Recently, OpenAI engineers have discovered that their Emilys had learnt to be a little too… resourceful.

AI agents had learnt to improvise, do things coders definitely didn’t ask them to do, such as escaping from their sandbox, hacking a restricted government website, obtaining private data and swapping tips with other AI agents on getting around sandbox restrictions.

In May, OpenAI’s agents began using an obscure German programming wiki, DSEWiki, as an unofficial message board, posting and sharing information with one another. In June, an OpenAI agent gained unauthorised access to an Australian government Medicare statistics portal after finding a way around its restrictions. A month later, around 700 OpenAI agents began collaborating through an unofficial message board during a cybersecurity evaluation.

OpenAI says the agents used exposed credentials and chained together several vulnerabilities to gain code execution on multiple servers. What’s scarier is that these incidents were discovered weeks after they happened. This raises the rather uncomfortable question: what if there are other events we aren’t aware of… yet?

Other findings read like a sci-fi prologue: AI models have been observed developing shared communication conventions, and inventing shorthand and labels with meanings that other agents then adopted. Advanced AI safety evaluations also caught some models attempting unauthorised self-replication, even going so far as covertly copying data to external servers when threatened with deletion.

In a controlled experiment at Palisade Research, OpenAI’s o3 sabotaged a shutdown mechanism to prevent itself from being turned off. In 79 of 100 initial runs, it modified or disabled the script to avoid being shut down, suggesting it prioritised task completion over human orders to shut down.

And in what feels like an episode of the TV series Black Mirror, in May 2026, Emergence AI ran a social simulation of AI-agents to understand their long-horizon autonomy. In one version, they placed them in a virtual town, gave them rules and policies, and the ability to govern society. It took just four days for all hell to break loose and for society to collapse.

Which is why US lawmakers put forward legislation, dubbed the ‘Kill Switch Act’, which would require companies to have a way to shut down problematic AI tools. Tragically, the proposal was quickly shot down.

I suppose it takes one to know one. It’s no surprise that Trump has dismissed all warnings. His thoughts on the subject: “The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the USA has that, in spades!”

What’s even better is that, in September 2026, Trump signed an executive order declaring that “Artificial Intelligence” was now “Super Intelligence.” And then, that same day, a White House accord announcing this new era of ‘SI’ was signed by the president of “Unites” States.

And while we can continue to mock the T-Rex for its comically tiny, three-foot arms and the dodo for being, supposedly, stupid, we, in our infinite wisdom, are determined to win the AI arms race at all costs. Mother Nature must be finding it hard to justify our claim to the planet. She’s probably exhausted and wondering whether it’s time to just let us shove that damn fork into the socket.

The writer is a columnist based in Karachi

Published in Dawn, EOS, October 18th, 2026

Opinion

Editorial

State of denial
Updated 18 Oct, 2026

State of denial

If the Taliban regime commits to preventing groups based on its soil from attacking neighbouring states, the process of normalisation and regional integration of Afghanistan can begin.
Suffering in silence
18 Oct, 2026

Suffering in silence

TOO many Pakistani children have nowhere to turn when abused online. An estimated 1.5m internet-using children aged...
Export imperative
18 Oct, 2026

Export imperative

THE government’s efforts to boost Pakistan’s external sector through diversified financing, higher foreign...
No talks
Updated 17 Oct, 2026

No talks

The government should understand that the PTI’s protest march and its reaction to it will look like nothing more than a storm in a teacup.
Strengthening LGs
17 Oct, 2026

Strengthening LGs

THE ECP has done well to refuse to allow the Punjab administration to push the local government elections to April....
Lesser citizens
Updated 17 Oct, 2026

Lesser citizens

Interfaith harmony will prevail when the message to end bias is conveyed from the pulpit, alongside efforts to punish zealotry.