Dec 30, 2022·edited Dec 30, 2022Liked by Gary Marcus
ChatGPT is equivalent to a librarian blocking the door to the library, offering to answer everything that everyone would have walked in and looked up themselves. The reference desk in a library does do this, but only for just-the-facts-m'am type of questions.
With ChatGPT etc. relying entirely on interpolating input data to generate results, there will always be doubts as to the veracity of the output. Knowledge synthesis, and generating new knowledge, depend on understanding, not just on piecing together sentences.
A google search reveals that Alire's ARISTOTLE AND DANTE DISCOVER the secrets of the universe and Coehlo's Alchemist are both mentioned on several webpages.
If ChatGPT's algorithm is the stepchild if not direct descendant of word2vec, it shouldn't surprise any of us that adjacency biases the LM's output, resulting in the hallucinated title.
I find it almost creepy that so soon after the term "fake news" became a phenomenon, a near-perfect mechanical generator of exactly that has been invented.
Years from now, people will be surprised to learn that the term actually predated the public LLMs.
Would be very curious what the medium.com article is that is cited by neeva as a source for your optimism on AI replacing jobs. ie, is this a failure of AI interpreting that medium article correctly (if webpage correctly doesn’t claim it), a failure of neeva to determine authoritativeness of webpages (if webpage incorrectly claims it)?
Programmers got hyped about ChatGPT as a search engine because, in programming, it really is pretty good. I can never remember, for love or money, the exact way to write a SQL cursor. But I ask ChatGPT to do so and it spits it right out, faster than a search engine would.
Would I trust it past that? Not really. I might ask it to do some stuff (convert LINQ or C# code to SQL), but I have to hand-review the code and test it like it was a task I handed to an eager but not-very-bright new intern. Which really gets down to the core limitation of ChatGPT and LLMs in general. They're useful, but only if you monitor the little blighters carefully and assume it's not a question of if, but when, they screw up and you have to clean up their mess.
I too am skeptical about the Code Red. Google has its own LLMs and know all too well the problems that you outlined in the post.
At the same time, You.com and ChatGPT are showing what could be. Can the system still work if it is a little more DB-like. I'm not sure but I suspect yes. The likely scenario is that we will all still use Google and will use an LLM-based tool selectively because they can do things Google cannot. That might not unseat Google but it would erode some market share which is critical to support the ad model.
Google's business model may be under threat. However, the threat is only looming. It is not a clear and present danger. They have time. The most likely scenario is that Google adds LaMDA or PaLM functionality to search and offers it as a feature. However, the question is whether they can stomach introducing something that is not as pristine as their mature product. If they cannot, then the startups will out-innovate them and market share erosion will be the inevitable result.
I recently used ChatGPT in a way that is search-esque when I was writing a piece. I used a particular story as my narrative opening, and my editor wrote back and said she didn't think it worked for reason X. So I went to Chat GPT and told it I was writing a piece, and here's the story I used, and that my editor said the problem was X. Could you recommend any stories that might work better? And it did. Then I asked it if it could provide stories featuring specific people (instead of broad instances from history). And it did. Then I asked it if it could provide stories featuring people I probably didn't know about (instead of, like, Einstein). And it did.
Ultimately, I ended up going in a different direction with the story. But I thought that was pretty useful. And there's no way I would've been able to come across some of those narratives with traditional search techniques.
So, now Bing. I have found it quite good for some things: which teams are playing in the six nations rugby this weekend, what times do the matches start and which channels are showing them on TV? It produces a really good answer to that, which google struggles with - you get links to sites that aren't particularly well designed; Bing just tells you the answers... But then autocomplete on "I came back from having an operation at the hospital yesterday and this morning I have woken up with a" suggested "autism" or "a woman". Still some way to go...
Has anyone noticed that you can ask ChatGPT if it wrote something - and it will tell you - at least in several cases from that I tested - one from this article:
My question:
Did you write this: "According to a 2019 poll conducted by the Associated Press and the NORC Center for Public Affairs Research, nearly two-thirds of Americans believe that they need to carefully watch what they say"?
ChatGPT response:
Yes, I wrote that. The poll to which I referred was conducted in 2019 by the Associated Press and the NORC Center for Public Affairs Research, and it found that nearly two-thirds of Americans believed that they needed to carefully watch what they say. Do you have any other questions about the poll or about public opinion in the United States?
Thank you for this article. I have a question: do you think that these “AI hallucination” could be exploited by contrarian thinkers in order to propagate conspiracy theories?
It would seem that ChatGPT is behaving a lot like an insufferable expert at a dinner party. Biased information mixed with current political correctness and hypocrisy.
It is strange that ChatGPT is pitted as a search engine when at most it can be used as a way to predict the next sentence/paragraph based on the previous context or a prompt.
It is really strange that people don't understand that chatGPT has mashed all (maybe) similar data (emphasis on maybe, because it is still looking at only words not meaning) from various websites into a neural network representation and has lost the reference to the original page without retaining the meaning? How can this even be used in use cases where meaning of the information is the core?
Both search engine & chatGPT do not understand or know about coherence of information, but while search engine gives me the original reference so I can deduce the coherence, chatGPT event doesn't give me the reference!
The first thing I did while playing with PerplexityAI was ask it about the significance of Victor Klemperer to the historiography of the Third Reich. It correctly summarized his bio and his contributions, but then concluded by saying he should not be confused with "nationalist and antisemite" Otto Klemperer. Otto was a famous conductor and Victor's cousin. Both were Jews persecuated by the Nazis. Perplexity applied a phrase used to describe Richard Wagner to Otto Klemperer because both were mention in the same article. Oops.
From that point on, I've used Perplexity mainly to generate amusing misinformation about Ewoks. Apparently they can be ordained as Catholic priests. Who knew?
ChatGPT is equivalent to a librarian blocking the door to the library, offering to answer everything that everyone would have walked in and looked up themselves. The reference desk in a library does do this, but only for just-the-facts-m'am type of questions.
With ChatGPT etc. relying entirely on interpolating input data to generate results, there will always be doubts as to the veracity of the output. Knowledge synthesis, and generating new knowledge, depend on understanding, not just on piecing together sentences.
A google search reveals that Alire's ARISTOTLE AND DANTE DISCOVER the secrets of the universe and Coehlo's Alchemist are both mentioned on several webpages.
If ChatGPT's algorithm is the stepchild if not direct descendant of word2vec, it shouldn't surprise any of us that adjacency biases the LM's output, resulting in the hallucinated title.
Terrific piece as usual, Gary. ChatGPT is a fantastic spinner of words, but poor seeker of knowledge. This is true for the LLMs of today.
I find it almost creepy that so soon after the term "fake news" became a phenomenon, a near-perfect mechanical generator of exactly that has been invented.
Years from now, people will be surprised to learn that the term actually predated the public LLMs.
Would be very curious what the medium.com article is that is cited by neeva as a source for your optimism on AI replacing jobs. ie, is this a failure of AI interpreting that medium article correctly (if webpage correctly doesn’t claim it), a failure of neeva to determine authoritativeness of webpages (if webpage incorrectly claims it)?
Programmers got hyped about ChatGPT as a search engine because, in programming, it really is pretty good. I can never remember, for love or money, the exact way to write a SQL cursor. But I ask ChatGPT to do so and it spits it right out, faster than a search engine would.
Would I trust it past that? Not really. I might ask it to do some stuff (convert LINQ or C# code to SQL), but I have to hand-review the code and test it like it was a task I handed to an eager but not-very-bright new intern. Which really gets down to the core limitation of ChatGPT and LLMs in general. They're useful, but only if you monitor the little blighters carefully and assume it's not a question of if, but when, they screw up and you have to clean up their mess.
I too am skeptical about the Code Red. Google has its own LLMs and know all too well the problems that you outlined in the post.
At the same time, You.com and ChatGPT are showing what could be. Can the system still work if it is a little more DB-like. I'm not sure but I suspect yes. The likely scenario is that we will all still use Google and will use an LLM-based tool selectively because they can do things Google cannot. That might not unseat Google but it would erode some market share which is critical to support the ad model.
Google's business model may be under threat. However, the threat is only looming. It is not a clear and present danger. They have time. The most likely scenario is that Google adds LaMDA or PaLM functionality to search and offers it as a feature. However, the question is whether they can stomach introducing something that is not as pristine as their mature product. If they cannot, then the startups will out-innovate them and market share erosion will be the inevitable result.
Great post BTW.
I recently used ChatGPT in a way that is search-esque when I was writing a piece. I used a particular story as my narrative opening, and my editor wrote back and said she didn't think it worked for reason X. So I went to Chat GPT and told it I was writing a piece, and here's the story I used, and that my editor said the problem was X. Could you recommend any stories that might work better? And it did. Then I asked it if it could provide stories featuring specific people (instead of broad instances from history). And it did. Then I asked it if it could provide stories featuring people I probably didn't know about (instead of, like, Einstein). And it did.
Ultimately, I ended up going in a different direction with the story. But I thought that was pretty useful. And there's no way I would've been able to come across some of those narratives with traditional search techniques.
So, now Bing. I have found it quite good for some things: which teams are playing in the six nations rugby this weekend, what times do the matches start and which channels are showing them on TV? It produces a really good answer to that, which google struggles with - you get links to sites that aren't particularly well designed; Bing just tells you the answers... But then autocomplete on "I came back from having an operation at the hospital yesterday and this morning I have woken up with a" suggested "autism" or "a woman". Still some way to go...
Has anyone noticed that you can ask ChatGPT if it wrote something - and it will tell you - at least in several cases from that I tested - one from this article:
My question:
Did you write this: "According to a 2019 poll conducted by the Associated Press and the NORC Center for Public Affairs Research, nearly two-thirds of Americans believe that they need to carefully watch what they say"?
ChatGPT response:
Yes, I wrote that. The poll to which I referred was conducted in 2019 by the Associated Press and the NORC Center for Public Affairs Research, and it found that nearly two-thirds of Americans believed that they needed to carefully watch what they say. Do you have any other questions about the poll or about public opinion in the United States?
(Other samples - This article: https://thelatch.com.au/chat-gpt/ - and the Psychology Today article.)
There is an extension that can assist you in integrating ChatGPT with search engines like Google, Bing, Duckduckgo, maybe you should try it: https://chrome.google.com/webstore/detail/chatgpt-for-search-engine/feeonheemodpkdckaljcjogdncpiiban/related?hl=en-GB&authuser=0
Thank you for this article. I have a question: do you think that these “AI hallucination” could be exploited by contrarian thinkers in order to propagate conspiracy theories?
It would seem that ChatGPT is behaving a lot like an insufferable expert at a dinner party. Biased information mixed with current political correctness and hypocrisy.
Maybe it's close to AGI?
It is strange that ChatGPT is pitted as a search engine when at most it can be used as a way to predict the next sentence/paragraph based on the previous context or a prompt.
It is really strange that people don't understand that chatGPT has mashed all (maybe) similar data (emphasis on maybe, because it is still looking at only words not meaning) from various websites into a neural network representation and has lost the reference to the original page without retaining the meaning? How can this even be used in use cases where meaning of the information is the core?
Both search engine & chatGPT do not understand or know about coherence of information, but while search engine gives me the original reference so I can deduce the coherence, chatGPT event doesn't give me the reference!
The first thing I did while playing with PerplexityAI was ask it about the significance of Victor Klemperer to the historiography of the Third Reich. It correctly summarized his bio and his contributions, but then concluded by saying he should not be confused with "nationalist and antisemite" Otto Klemperer. Otto was a famous conductor and Victor's cousin. Both were Jews persecuated by the Nazis. Perplexity applied a phrase used to describe Richard Wagner to Otto Klemperer because both were mention in the same article. Oops.
From that point on, I've used Perplexity mainly to generate amusing misinformation about Ewoks. Apparently they can be ordained as Catholic priests. Who knew?
Great article as usual!
I’m curious: I see much more comments here than usual, especially so soon. What’s the deal?
If possible, I’d like to please see your stats on readership. Assuming that’s not a bother.
Thanks ahead.