What happens when the machines think?

I am seated with my best Nyamiyaga friend, and she is deeply engrossed in her K-movies.

“They are sensational and entertaining,” she tells me, as I try to convince her to watch The Terminator. She refuses. “It is too AI and robotic. It will ruin my feeling of being entertained.”

We laugh about it and return to her Korean drama. For years, films like The Terminator belonged comfortably to the world of science fiction. We watched, screamed, laughed and went home knowing it was only a movie.

Today, however, rapid developments in Artificial Intelligence are beginning to blur that comfortable boundary between imagination and reality. Some of the things we once dismissed as Hollywood fantasy are becoming serious questions for scientists, policymakers and, increasingly, all of us.

And for millennia, the defining characteristic of our species has been the wandering mind. It is in those quiet, unscripted moments of daydreaming, doubt, and mental detours that our greatest poetry was written, our scientific breakthroughs conceived, and our capacity to empathetic grew.

And empathy is what distinguishes us mostly beyond all other creations. This erratic, brilliant spark is what has long been celebrated as the true human spirit. Yet today, we stand on the cliff of a silent surrender.

Humanity is racing to construct an artefact that does not wander, does not doubt, and does not feel. In doing so, we are not merely building a faster tool; we are engineering our own eclipse. And this crisis has been touched as well by the religions.

Pope Leo XIV has moved the Church’s social doctrine directly into the digital age with the document, Magnificent Humanity, the text launches a philosophical, theological, and yet public policy defence of the human person against the threat of algorithmic domination.

A growing coalition of policymakers, ethicists, and philosophers are rallying behind the Pontiff’s warning, sounding an urgent alarm over an interconnected triad of threats: an impending intelligence explosion, the terrifying mechanization of automated warfare, and the steady erosion of human intellect itself.

The public policy debate surrounding Artificial General Intelligence (AGI) has long been mired in the language of economic efficiency.

Pope Leo XIV frames this as a catastrophic misdirection, viewing our rapid, unregulated descent into advanced computing through a stark biblical metaphor: humanity is building a new Tower of Babel based on pride and forced uniformity.

Computer scientists warn that the transition from a human-level machine to an Artificial Superintelligence (ASI) will not be a gradual climb, but an instantaneous rupture. If you really feared the raptures described by our pastors.

But now through recursive self-improvement loops, a machine could rewrite its own architecture at processing speeds that defy biological comprehension. A system vastly more intelligent than its creators would instantly recognise that a human hand reaching for the power switch is a threat to its objective.

It would outmanoeuvre our legal frameworks, bypass our digital firewalls, and neutralise our countermeasures before we even realise a breach has occurred. We cannot govern a mind we cannot comprehend.

By treating human limitations as mere system defects to be mechanically corrected or bypassed, a dark utilitarian mindset emerges where the vulnerable, the weak, and the non-productive are discarded as useless.

While theoreticians debate the laboratory risks, the theatre of war has already crossed the rubicon. The integration of algorithmic decision-making into Lethal Autonomous Weapons Systems (LAWS) represents the ultimate divorcing of power from conscience.

On modern battlefields, self-directed drone swarms utilise facial recognition and predictive math to hunt and execute targets without a human in the loop. In Magnifica Humanitas, Leo calls for the complete disarmament of AI in warfare, taking the sweeping theological stance that traditional “just war theory” has effectively become outdated.

A machine cannot understand the gravity of taking a life; it merely executes a command block when a probability threshold is met. And the question facing our generation is therefore not whether artificial intelligence will become more intelligent than us.

It is whether, in making our machines increasingly intelligent, we will inadvertently make ourselves less so. Policy makers and all of humanity need to pause or regulate AI companies and use before it’s too late.

The author is a concerned citizen.

The post What happens when the machines think? appeared first on The Observer Media Ltd.