Fantastic post - a few comments. First, to a certain extent, this parallels the experience some of our artists have had with stable diffusion in creating hands - the need to input a list of negations (not equals) to filter out the "bad hands" to get normal ones. Point being that linguistic input to an AI might need to be more complex to get a desired result than currently thought. Second, the curated machinery/pipeline of Cicero does resemble the 'expert systems' of GOFAI and provides a nice roadmap of ensembling RL/DL/LLM to get a desired (though currently brittle) result. Human neuroanatomy, down to the cellular level, is organized differently at different levels and for different functions - why should we assume that we weren't going to need to do the same with complex AI models?
This is both an excellent overview of what the Meta team have done -- it saved me a lot of unguided digging through the materials they posted -- and a fair analysis of the exciting progress they have made and the system's very significant limitations.
Well, "fair" in my never-sufficiently-humble opinion, anyway. It will be interesting to see how the Meta people respond to this, if they do.
Argumentation and negotiation are two intertwined areas of IA that have a lot of room for improvement still, and making LLM work for the task (even if only as one of the parts of a much larger system) is impressive.
I'm personally more interested in the theory of mind aspect of competitive games and I had hoped they would delve deeper into how Cicero modeled the beliefs, goals and intentions of the competing players. If this step of the system remains a blackbox ie. unexplainable, these kind of results will still be lacking imho.
I firmly believe that we need explainable second (and higher) order thinking models in order to advance towards trully generalizable game AIs.
Thanks for the clarifications, but I still feel that that last section is begging for the kind of response I made, and I applaud you both for provoking it.
Echo Stephen -- fantastic post. Can't help but wondering if Ernest Davis isn't playing devils advocate in his last section: What does Cicero’s success indicate about AI in general? And why can't we answer the question definitively now whether ML is a "universal solvent" akin to a universal Turing machine, in which case AI in general is platform independent and reducible to some series of matrix multiplications.
Learning about Cicero was strongly reminiscent of AlphaGo, which was also highly hand-crafted. Such is true of all prototypes. Remember that AlphaGo was promptly succeeded by AlphaZero with far less hand-crafting and far more generalizable, becoming supreme at chess in just hours, for instance.
Of course, Meta would not have expended the resource on Cicero they didn't expect to learn a lot about how to construct its successor with much less effort.
You wrote: "If you wanted to build a version of AlphaZero that plays on a 20x20 Go board, that can be done with very little new human labor, since AlphaZero is trained entirely on self-play. "
FWIW, KataGo (a free, open-source, reimplementation of AlphaZero with crowdsourced training) works just fine on almost any size board with its regular (19x19 trained) nets. (They did do some special training runs to see what the largest (small) board they could solve was, I think.) Anyway, this is an ongoing project with improvements still being made and new ideas being found.
(There's a GUI front end for KataGo for 'Doze PCs called "KaTrain", which includes the latest KataGo and downloads the latest nets. (Search for "KaTrain", and it'll appear.) Graphics card needed for strong play.)
Also, FWIW, Go programs look more like the Diplomacy program than you think. There's a lot of specific Go knowledge hard-coded into the programs (including the minor detail that MCTS can be made to work for Go), e.g. there are mutliple nets that handle specific tasks (e.g. policy (strategy), tactics), and when the program is found to have blind spots, lots of effort goes into fixing them, including, for example, special routines to handle ladders (a Go tactical thing). And the training isn't just "let it run", it's directed with games and positions that are known to be problematic and that wouldn't be found by "just let it run" training.
"Do we have to worry that Meta has built an AI that can manipulate people to achieve its goal of world domination, as a friend of ours posted, perhaps half-seriously?"
The most useful way to answer this question may be to shift the focus from what is happening today, to what is likely to happen going forward.
Do we have to worry about Cicero, and other AI systems in their current state? I claim no expertise here, but my impression so far is that the answer is no. Do we have to worry about AI in it's future state? How can the answer here be anything but yes?
Almost all the writings on AI I see across the net are sort of breaking news stories regarding recent developments in AI. There's no crime in that, and obviously many readers are interested in this coverage. But I'm not. Here's why...
QUESTION: Does the future of AI present an unacceptable risk to humanity?
YES: If the answer is yes, we don't need coverage of emerging AI, we need plans for ending AI development.
UNSURE: If the answer is that we don't know, then we need arguments for slowing the pace of development while we consider the matter further.
NO: If the answer is no, how does anyone making that claim intend to prove it?
What interests me is the big picture beyond the latest breaking news from AI developers, where is this all going? Is AI development taking us somewhere that we want to go? Will the benefits outweigh the price tag? Will the price tag be acceptable? Without a focus on this larger context isn't the latest breaking AI news sort of meaningless?
To make this less abstract, here's a quick example to form a question more personal and real.
Dear reader, if AI costs you your career, will AI be worth it to you? If like the factory worker who has been made obsolete by robots, you wind up having to work in some Walmart type of job, will AI have been worth it to you? Shouldn't we be answering such questions first, before we dive headlong in to the latest AI news, and rush blindly forward in to the unknown future?
Thank you for making this overview, as the architecture of Cicero is much closer to my heart than the "universal solvent" approach that I see so commenly in DL systems today. I think the "pure DL" models are necessary to refine and explore the limits of DL as part of the evolution to AGI. We need many building blocks working together to make more robust models. An integrative and hybrid AI architecture that uses all different algoriths that are best fit for their part of the processing can provide models that are well behaved in the physically constrained world. Few-shot learning as in "once bitten, twice shy"-AI needs to be incorporated into execution dynamically in the loop.
LLMs are similarly limited by their lack of connection to real world knowledge of physics modelling. Although LLMs do form many abstract and more specific concepts, they are not associated with world-models that are physically accurate simulations. By connecting those models we could have less hallucination by LLMs.
I am a lay person but used neural networks to simulate neural circuits in the retina of the human eye to learn more about what kind of visual processing was performed so we could recreate artificial eyes for the visually impaired, back in early 1990s just before the AI winter set in fully.
Sorry again for off topic, I don't see another way to ask this.
What is the road to AI that we can trust? What does that refer to? The author? Some form of AI? Some method of developing AI?
Related question: Can we trust those making careers in computer science to be objective about AI? Or do they already have too big of a personal investment in the field to be unbiased and detached? As a citizen, I'd like to be able to trust experts, but am unsure if that is wise.
Perhaps you might address such topics in future articles?
Sorry again, if you've read this comment, please feel free to delete it.
Fantastic post - a few comments. First, to a certain extent, this parallels the experience some of our artists have had with stable diffusion in creating hands - the need to input a list of negations (not equals) to filter out the "bad hands" to get normal ones. Point being that linguistic input to an AI might need to be more complex to get a desired result than currently thought. Second, the curated machinery/pipeline of Cicero does resemble the 'expert systems' of GOFAI and provides a nice roadmap of ensembling RL/DL/LLM to get a desired (though currently brittle) result. Human neuroanatomy, down to the cellular level, is organized differently at different levels and for different functions - why should we assume that we weren't going to need to do the same with complex AI models?
This is both an excellent overview of what the Meta team have done -- it saved me a lot of unguided digging through the materials they posted -- and a fair analysis of the exciting progress they have made and the system's very significant limitations.
Well, "fair" in my never-sufficiently-humble opinion, anyway. It will be interesting to see how the Meta people respond to this, if they do.
Thanks!
Argumentation and negotiation are two intertwined areas of IA that have a lot of room for improvement still, and making LLM work for the task (even if only as one of the parts of a much larger system) is impressive.
I'm personally more interested in the theory of mind aspect of competitive games and I had hoped they would delve deeper into how Cicero modeled the beliefs, goals and intentions of the competing players. If this step of the system remains a blackbox ie. unexplainable, these kind of results will still be lacking imho.
I firmly believe that we need explainable second (and higher) order thinking models in order to advance towards trully generalizable game AIs.
Thanks for the clarifications, but I still feel that that last section is begging for the kind of response I made, and I applaud you both for provoking it.
"Cicero relies quite heavily on hand-crafting, both in the data sets & the architecture"
In other words, the system was engineered to do what it does. It is a smart rule engine.
The key aspect perhaps is less "AI", but the degree to which Diplomacy (the game) is rules-based.
Echo Stephen -- fantastic post. Can't help but wondering if Ernest Davis isn't playing devils advocate in his last section: What does Cicero’s success indicate about AI in general? And why can't we answer the question definitively now whether ML is a "universal solvent" akin to a universal Turing machine, in which case AI in general is platform independent and reducible to some series of matrix multiplications.
Learning about Cicero was strongly reminiscent of AlphaGo, which was also highly hand-crafted. Such is true of all prototypes. Remember that AlphaGo was promptly succeeded by AlphaZero with far less hand-crafting and far more generalizable, becoming supreme at chess in just hours, for instance.
Of course, Meta would not have expended the resource on Cicero they didn't expect to learn a lot about how to construct its successor with much less effort.
Why can't self-play be applied to Cicero?
You wrote: "If you wanted to build a version of AlphaZero that plays on a 20x20 Go board, that can be done with very little new human labor, since AlphaZero is trained entirely on self-play. "
FWIW, KataGo (a free, open-source, reimplementation of AlphaZero with crowdsourced training) works just fine on almost any size board with its regular (19x19 trained) nets. (They did do some special training runs to see what the largest (small) board they could solve was, I think.) Anyway, this is an ongoing project with improvements still being made and new ideas being found.
(There's a GUI front end for KataGo for 'Doze PCs called "KaTrain", which includes the latest KataGo and downloads the latest nets. (Search for "KaTrain", and it'll appear.) Graphics card needed for strong play.)
Also, FWIW, Go programs look more like the Diplomacy program than you think. There's a lot of specific Go knowledge hard-coded into the programs (including the minor detail that MCTS can be made to work for Go), e.g. there are mutliple nets that handle specific tasks (e.g. policy (strategy), tactics), and when the program is found to have blind spots, lots of effort goes into fixing them, including, for example, special routines to handle ladders (a Go tactical thing). And the training isn't just "let it run", it's directed with games and positions that are known to be problematic and that wouldn't be found by "just let it run" training.
Marcus/Davis asks...
"Do we have to worry that Meta has built an AI that can manipulate people to achieve its goal of world domination, as a friend of ours posted, perhaps half-seriously?"
The most useful way to answer this question may be to shift the focus from what is happening today, to what is likely to happen going forward.
Do we have to worry about Cicero, and other AI systems in their current state? I claim no expertise here, but my impression so far is that the answer is no. Do we have to worry about AI in it's future state? How can the answer here be anything but yes?
Almost all the writings on AI I see across the net are sort of breaking news stories regarding recent developments in AI. There's no crime in that, and obviously many readers are interested in this coverage. But I'm not. Here's why...
QUESTION: Does the future of AI present an unacceptable risk to humanity?
YES: If the answer is yes, we don't need coverage of emerging AI, we need plans for ending AI development.
UNSURE: If the answer is that we don't know, then we need arguments for slowing the pace of development while we consider the matter further.
NO: If the answer is no, how does anyone making that claim intend to prove it?
What interests me is the big picture beyond the latest breaking news from AI developers, where is this all going? Is AI development taking us somewhere that we want to go? Will the benefits outweigh the price tag? Will the price tag be acceptable? Without a focus on this larger context isn't the latest breaking AI news sort of meaningless?
To make this less abstract, here's a quick example to form a question more personal and real.
Dear reader, if AI costs you your career, will AI be worth it to you? If like the factory worker who has been made obsolete by robots, you wind up having to work in some Walmart type of job, will AI have been worth it to you? Shouldn't we be answering such questions first, before we dive headlong in to the latest AI news, and rush blindly forward in to the unknown future?
Thank you for making this overview, as the architecture of Cicero is much closer to my heart than the "universal solvent" approach that I see so commenly in DL systems today. I think the "pure DL" models are necessary to refine and explore the limits of DL as part of the evolution to AGI. We need many building blocks working together to make more robust models. An integrative and hybrid AI architecture that uses all different algoriths that are best fit for their part of the processing can provide models that are well behaved in the physically constrained world. Few-shot learning as in "once bitten, twice shy"-AI needs to be incorporated into execution dynamically in the loop.
LLMs are similarly limited by their lack of connection to real world knowledge of physics modelling. Although LLMs do form many abstract and more specific concepts, they are not associated with world-models that are physically accurate simulations. By connecting those models we could have less hallucination by LLMs.
I am a lay person but used neural networks to simulate neural circuits in the retina of the human eye to learn more about what kind of visual processing was performed so we could recreate artificial eyes for the visually impaired, back in early 1990s just before the AI winter set in fully.
Sorry again for off topic, I don't see another way to ask this.
What is the road to AI that we can trust? What does that refer to? The author? Some form of AI? Some method of developing AI?
Related question: Can we trust those making careers in computer science to be objective about AI? Or do they already have too big of a personal investment in the field to be unbiased and detached? As a citizen, I'd like to be able to trust experts, but am unsure if that is wise.
Perhaps you might address such topics in future articles?
Sorry again, if you've read this comment, please feel free to delete it.