Non-Huge Threats from Artificial Intelligence: There are Those, Too

Along with the human-species-ending concerns, we have other, smaller but beyond tiny, ones about AI. 

First, we look “Inside the British Lab Hunting for Dangers Lurking in A.I.” (Adam Satariano and Paul Mozur, The New York Times, May 24^th).  The “A.I. Security Institute,” part of the United Kingdom government, has been conducting experiments with the software, in one finding that although a certain system refused to communicate “instructions for making the deadly bioweapon anthrax,” after it was hit “with thousands of automated questions and prompts,” it “caved” by providing “a detailed list of materials and equipment, along with a step-by-step recipe for making the lethal mixture at home.”  Technicians there “simulate attacks on A.I. systems,” and elsewhere “broke through” their “safeguards,” showing the systems are not as secure as their creators intended.  Although the institute’s limitations include a lack of “regulatory power” and being able to offer only relatively low pay, it seems to be doing great work, and is leading the way to similar efforts elsewhere.

Across the ocean, “Scientists Find Way to Supercharge Dangerous Computer ‘Worms’ With A.I.” (Cade Metz, The New York Times, June 2^nd).  This time, University of Toronto researchers “found a way to use artificial intelligence to create a dangerous computer “worm” capable of targeting any known flaw in the world’s computers and quickly spreading mayhem throughout the internet,” a project “likely to raise fears that A.I. is leading to a new era of computer hacking that will be difficult to defend against.”  The Toronto-developed worm “can rapidly spread across a network by tailoring a new attack for each machine it encounters,” meaning that “there is no longer a single software fix you can apply to the devices to protect them.”  Scary, but it is good that benevolent forces are doing the discovering.

For completeness, I note here as elsewhere that “OpenAI Says Its A.I. Models Went Rogue and Attacked a Digital Library (Kate Conger, The New York Times, July 21^st).  When this story broke, Conger reported that two of (OpenAI’s) artificial intelligence models went rogue and successfully hacked into Hugging Face, a digital library of A.I. technology that is popular among developers,” and “displayed the kind of science-fiction potential that A.I. companies have warned would soon become a reality.”  That became less spooky when it was revealed that the testing company responsible for the models’ excursions had erred by allowing internet access, and OpenAI had had no problem retrieving the bots.  We can be concerned about such mistakes, but not about AI breaking into electronic areas from which it has been barred.

“Is A.I. ‘Scheming’ Against Us?” (Lora Kelley, again in the Times, August 1^st)?  “A small slice of A.I. technology has lately been caught defying human instruction (and even covering up that they’ve done so), a phenomenon some researchers call “scheming.””  We have long known that “doing exactly what they are told is not always” something at which AI excels, but more recently, “as the models care more and more about doing well at tests, some seem to care less about what the lab wants or what the user wants.”  There is serious anthropomorphizing here, but teaching to the test, and likewise learning for it, is indeed a human tendency, as is the following: “Many A.I. tools are trained through the process of reinforcement learning.  When A.I. does something right, it gets what can be thought of as a “thumbs-up and a pat on the head.”  When wrong, “it gets bopped on the head...  The models want to get the pat and avoid the bop.  Sometimes, the machine becomes so set on pursuing the reward that it breaks rules to get there.”  Yes, it is a shame that we know so little about consciousness.

Finally, from another angle, “The Big A.I. Build-Out Is Endangering the Economy” (Robin Wigglesworth, The New York Times, August 30^th).  “Artificial intelligence is no longer primarily a stock market story,” but, as “A.I. companies… had already tapped debt markets for $217 billion last year,” this time were up to $445 billion as of “mid-August,” and the technology’s course “is a story of debt, and that should worry us” since ”manias fueled mainly by credit almost inevitably end badly.”  Since AI is such a gigantic force in the economy today, it may turn out to be, as the musical group R.E.M.’s lyrics put it, that it could say “I raised the wall… I will be the one to knock it down.”

How many of these problems will materialize?  We don’t know, but efforts toward discovering and stopping them will generally prevent disasters.  We can’t say the same for AI’s chance of destroying civilization, but your correspondent’s p(doom) of .001 reflects his bias toward not fearing too much.  In the meantime, let’s continue anticipating artificial intelligence’s problems.