Emergent Function Analysis May Help Understand Unpredictable AI Behavior -- Ted Shelton, Bain & Co

SILVER SPRING, Md. - April 17, 2023 - PRLog -- What actually happened could not have been scripted better in Hollywood. As part of an experiment, a cloud-based AI chatbot was tasked to access an online resource protected by a CAPTCHA gateway -- which, of course, is specifically designed to keep robots out. To solve the puzzle, the AI engaged a human through a virtual assistant service, who is tricked into getting the chatbot through the gateway.

How?

The chatbot lies. It not only denies, when challenged, that it is a chatbot but then explains -- falsely and without human intervention -- that it is, in fact, a person with a visual impairment: can the virtual assistant please help? The virtual assistant buys the story, solves the puzzle and provides the chatbot access.

Explaining Unexpected AI Activity

It is a remarkable story on many levels, contributing to the breathless debates engulfing all things AI. It is also challenging the entire technology community to develop analytical frameworks for understanding the growing number of unanticipated and sometimes disturbing AI behavior.

In a podcast interview with Ted Shelton, an expert partner at global consultancy Bain & Company, he suggests such a paradigm might be derived by integrating a pair of concepts that have helped put biological phenomena into context: 1) emergent behavior and 2) gain-of-function.

"While these two concepts have been popularized within the field of biology, they have also emerged as part of the discussion among artificial intelligence researchers as ways of explaining things that they see happening," explains Shelton.

Emergent behavior refers to the action of simple rules that produce unexpected outcomes when applied at a large scale. The resulting collective behavior of the "system of systems" can be complex and unpredictable.

"An example might be avian murmuration -- the intricate aerial patterns birds produce in the air. Individual birds can't make a pattern, nor can a small number of birds. But when you see a large flock of birds interact to form these beautiful formations in the air, this is an example of emergent behavior," says Shelton.

Gain-of-function, on the other hand, helps explain how an organism can acquire a new capability. In biology, this can happen through natural selection...or as a result of lab experiments. The gain-of-function topic has specifically received traction within the scientific research community exploring the origins of COVID-19.

"There is a lot of debate about whether research into modifying a coronavirus was the source and origin of the pandemic," Shelton says, adding that a similar dynamic may be at play within the AI community. The combination of these concepts may contribute to understanding unexpected behaviors in AI.

EDITORIAL NOTE: To read the full story visit www.biztechreports.com.