Agreed. The fundamental flaw with LLMs is that their "internal world models", such as they have them, are extremely poor (very broad, possibly, but also very shallow). Poor world models means they have (at best) very poor understanding of any concept pertaining to the real world, and therefore (at best) very weak reasoning abilities pert…
Agreed. The fundamental flaw with LLMs is that their "internal world models", such as they have them, are extremely poor (very broad, possibly, but also very shallow). Poor world models means they have (at best) very poor understanding of any concept pertaining to the real world, and therefore (at best) very weak reasoning abilities pertaining to the real world. And so LLM-based cognition is and will always be severely limited.
True but I would go farther than just "poor world models" as that leaves open the possibility they will get better with more training data. Instead, the world models were never designed to model truth. That they produce truth more often than falsity is simply a by-product of the fact that the world's text they're trained on is biased toward truth. Perhaps "non-truth-based models".
The answers given by LLMs are like the outcome of a popularity contest.
But, unfortunately, sometimes the truth is not popular.
Computer “scientists” keep inventing the same flawed methods. Google’s Page Rank system for search was also designed on the basis of a popularity contest.
True but it's the most popular sequence of words, not the most popular opinion or belief. Big difference. Page Rank was very successful, just like LLMs are now, but both will be eclipsed easily by AI technology that understands what it reads.
LLM alone will be severely limited, yes. Good world models are important, where we can get them. In poorly defined areas, fitting a neural net to lots of data (beyond just text) will likely be as good as it gets for quite a while.
Agreed. The fundamental flaw with LLMs is that their "internal world models", such as they have them, are extremely poor (very broad, possibly, but also very shallow). Poor world models means they have (at best) very poor understanding of any concept pertaining to the real world, and therefore (at best) very weak reasoning abilities pertaining to the real world. And so LLM-based cognition is and will always be severely limited.
True but I would go farther than just "poor world models" as that leaves open the possibility they will get better with more training data. Instead, the world models were never designed to model truth. That they produce truth more often than falsity is simply a by-product of the fact that the world's text they're trained on is biased toward truth. Perhaps "non-truth-based models".
The answers given by LLMs are like the outcome of a popularity contest.
But, unfortunately, sometimes the truth is not popular.
Computer “scientists” keep inventing the same flawed methods. Google’s Page Rank system for search was also designed on the basis of a popularity contest.
True but it's the most popular sequence of words, not the most popular opinion or belief. Big difference. Page Rank was very successful, just like LLMs are now, but both will be eclipsed easily by AI technology that understands what it reads.
LLM alone will be severely limited, yes. Good world models are important, where we can get them. In poorly defined areas, fitting a neural net to lots of data (beyond just text) will likely be as good as it gets for quite a while.