CICERO: an AI that knows how to persuade humans to achieve its ends

Meta has designed a unique intelligent system called CICERO (in reference to the Roman orator Cicero). As other AIs have done before in chess, Go or poker, CICERO is able to confront and beat humans in the game Diplomacy. And it’s not trivial, because this strategy game requires “using language to negotiate, persuade and collaborate with people to achieve strategic goals, the way humans do”. Tested online at webDiplomacy.net, CICERO performed twice as well as the average score achieved by human players and has even become players’ preferred partner, according to Meta.

The firm speaks of prowess, as the game Diplomacy is complex. At each game turn, the players exchange messages and then decide their move. For an AI, this means the ability to understand the motivations and perspectives of other players, to plan and adjust strategies, and above all to seal alliances with human players, simply by conversing with them. CICERO can, for example, deduce that he will need the support of a certain player later in the game, developing a strategy to win his favor by evaluating his perspective. “If an agent cannot recognize that someone is probably bluffing or that a move might be considered aggressive, they will quickly lose the game,” Meta points out on the blog post.

Unprecedented combination of two technologies

CICERO is an eminently complex system. It is distinguished in particular by the rare combination of two artificial intelligence techniques: strategic reasoning (like systems for chess or Go) and natural language processing, GPT-3 or LaMDA style. On the conversation side, the designers started from a general language model (2.7 billion parameters) which they improved by training it on exchanges between players on webDiplomacy.net. On the strategy side, the training data has been annotated with in-game moves, so that the system generates dialogues corresponding to the movements it wishes to perform. To prevent CICERO’s decisions from being stereotyped and other players from taking advantage of them, the researchers also designed an algorithm that seeks to predict the strategy of other players based on the messages exchanged, and iteratively adjusts its own strategy based on these predictions. Lastly, to be more effective, Cicero knows how to speak in persuasive language and reveal his intentions where necessary, by explaining to his interlocutor for example what they would both gain with the proposed move.

“Good old AI”

For Gary Marcus, the CICERO system is a marvel of integration of language and action. A rare compliment from this AI figure who is very critical of current developments. “Strikingly, and in opposition to much of the zeitgeist, Cicero relies heavily on craftsmanship, both in datasets and in architecture; in this sense, it is more reminiscent of classic “good old AI” than deep learning systems which tend to be less structured and less suited to particular problems. There’s a lot more innateness here than we’ve typically seen in recent AI systems,” Marcus explains on his blog.

Social engineering

However, CICERO poses the question of social engineering. On Twitter, its designers are pleased that players have been tricked into thinking they are talking to a human. And in their blog, they imagine innovative uses of their system, like the ability to converse with a person to teach them a skill, or making a video game understand players’ motivations and help them accomplish their mission.

But we could also imagine less virtuous uses, such as deploying bots capable of leading conversations on social networks to convince people to buy such and such a product, to vote for a certain candidate, to get vaccinated or, on the contrary. to refuse vaccination, etc. One could also imagine AIs that deceive users about their intentions or the accuracy of their prognosis if this serves the purpose for which they were programmed…

Cicero or Machiavelli?

CICERO’s capabilities are also cause for concern in the longer term. Since intelligent systems are devoid of common sense, some specialists, including Stuart Russell, worry that by wanting to do well, AIs will make unthinkable and devastating decisions. Their solution would be to have the AIs dialogue with the humans, make sure they have understood what is being asked of them, and that before acting they explain what they have in mind. intention to do. In this context, CICERO’s ability to persuade his interlocutors to achieve his ends could negate the usefulness of a dialogue between humans and machines…

Similar Posts

Leave a Reply

Your email address will not be published. Required fields are marked *