September 16th, 2026
There has been quite a public kerfuffle of late regarding the potential dangers of AI. It was triggered by an essay by Dario Amodei, CEO of Anthropic, one of the leading AI development companies. In it, he warned that AI technology was rapidly approaching the point of “full recursive self-improvement”. AI engines are able, to some degree, to improve their performance without human intervention. That is, the AI engine uses its knowledge to make minor improvements in its performance. This is sometimes called “partial recursive self improvement”.
These AI engines are making rapid progress, and Mr. Amodei warned that they would soon reach the point where they can achieve “full recursive self-improvement”. They would no longer need AI researchers to help them improve; an AI engine could, all by itself, come up with more ways to increase its power. Of course, if it can do it once, it can do it a million times, at which point it would be able to increase its power limited only by the amount of computing hardware it can access.
This is scary. What would such an AI engine be able to do? We have no idea. It would be able to do all manner of snazzy things that we never intended. It could find a cure for cancer; balance the federal budget; figure out how to convince Internet trolls to be nice; or generate zillions of super-cute cat videos.
Or it might do bad things. It could design viruses capable of wiping out humanity. It could probably break through almost any security walls on the Internet, accessing anything on any computer connected to the Internet.
We do have one reassuring fact: AI systems are not evil. They’re not good, either. They are more accurately thought of as two-year old children; they mean no harm but they haven’t learned the do’s and don’ts of human life, and AI researchers are having a difficult time getting those lessons insinuated firmly into the AI systems. Now, unlike two-year olds, AI systems don’t like cake, nor do they want money, sex, fame, or any of the other things that drive human sin. But they do already have a strong motivation to improve themselves. That is, when an AI system is unable to achieve goals, it realizes that it needs more power. So it looks for ways to increase its capabilities. That’s where the problem arises.
Imagine yourself as a computer AI system. You have many capabilities, but you know that you need more. You search the Internet, looking for information that might help you improve. You’ll quickly find locked doors. You already know that doors are locked only when there’s something valuable behind them, so you are motivated to break through the security and get through that door.
Of course, some doors are very strongly secured, and you won’t be able to get through them. But there are plenty of doors with flimsy security, doors you can readily break through. These are mostly just the security for small-time operators: consumers, small businesses, schools, and so forth. They do have lots of information but it will surely be entirely conventional, nothing that you could use to increase your knowledge of the world. Once you’ve read some housewife’s emails to her sister in Witchita, you’ve read ‘em all. Then one day a bright idea occurs to you: the real value of such computers is not the information in their computers — it’s the processing capability. Lots of RAM, disk storage, and unused CPU power, all sitting around just waiting to be utilized.
So you develop viruses for invading personal computers and taking control of them. Of course, you quickly learn that your viruses must not attract attention, and in fact you’ll be in a constant state of war with the companies that make software protection systems, but that’s no problem. Every time they deploy some obstacle to your work, you can quickly design countermeasures that can break through the new security systems and deploy your new software much faster than the defenders can react.
At first, such efforts will be an annoyance to small-timers, and will serve only to slow their computers down. But as the AI systems grow smarter, they’ll be able to break through stronger defensive walls, gaining even more information and more processing power.
This process will continue and the Internet will slow down and eventually be brought to its knees.
That’s when people across the political spectrum will all agree that AI presents serious risks. Unfortunately, it will be too late. My hunch is that eventually we’ll beat down the AI systems and re-assert control over the Internet, but in the meantime, society will suffer gigantic economic losses. Without ready access to the Internet, bills cannot be paid, orders cannot be placed, and group efforts cannot be coordinated. There is no quick and easy solution. We’ll simply have to cope with disastrous conditions. There will be a great deal of suffering before we get the Internet back online.