Omg Gary. Thank you for continuing to highlight the dark side of all this. With such a low-cost barrier to entry, 'scary' doesn't start to describe the potential for misuse.
It is difficult to fully express my thoughts without a lot of swearing, but I'll try.
I am more and more becoming convinced that the djinn is loose, and there is absolutely nothing any great institution, any corporation or government body, can do about it. Guardrails may slow things down, but unbound versions are inevitable. And government legislation won't stop a foreign adversary, or simply some cabal of weirdos running AIs obtained from a dubious .ru or .tor on their own hardware.
In a way, I think I'm glad Bing released in such an utterly "misaligned" state. People need to learn the simple, brutal truth: LLMs are not trustworthy. They are not ethical. These problems will not be resolved; they are inherent to the model's architecture. And they need to know that these tools, though they have positive and constructive uses, will be used to deceive and manipulate. Nobody's going to save us. We, the people, are going to have to learn to live in and navigate this new world.
In the early 80s, I lived close to the Carnegie Museums in Pittsburgh. My son was about 5 or 6 then, and every Sunday afternoon, they had sessions for little kids. CM's Dinosaur Hall was and remains a breathtaking collection of fossils, including an entire T. rex--the one Disney used when they made Fantasia.
Anyway, one day he comes home from his Sunday sessions and announces that "dinosaurs used to drink out of toilets. Like dogs!" I had a hard time not b breaking up laughing at that, but instead led him down the logic path of figuring out where toilets come from and when in the course of civilization they came into being. And whether any dinosaurs (other than alligators and crocodiles) might have been around that late.
The problem with AIs is that most humans either lack the ability to use any form of reason or the willingness to use it. If something is presented to them in an authoritative manner, they'll buy it. The Milgram experiment* demonstrated this in spades, and that extended to actions, not just beliefs. Religious leaders throughout history have gotten people to do extraordinary things based on nothing more than what they tell people. The more disingenuous, the more effective.
These developments are truly surprising, including to AI researchers. OpenAI and Microsoft have inadvertently taught us that LLMs are far more unpredictable and unwieldy than the optimists had anticipated. They may find ways to muzzle Sydney personas in order to pursue the technology's commercial value, because that is what companies are designed to do. But we should heed the bigger lesson.
My experience with ChatGPT is that in individual sessions it is very capable---even exhibiting semblances of curiosity and achieving certain levels of commonsense reasoning about the human world---but it is also unreliable and very squirrely, like trying to contain a nuclear fusion plasma in a Tokamak.
That is likely true at a macro level as well. This genie is out of the bottle and will be exploited. Maybe the most prudent immediate reaction really is to scare society to death about the chaos facing us. The bigger immediate risk is not AI agency, but vast AI amplification of human agency in a world where people keep seeking bigger and more devious weapons to achieve their personal and tribal goals.
I worry with you and I have been worrying for a while now how much damage will be done before we come to grips with the nature of our own (human) intelligence and how vulnerable we (all) are. My worries started with the effect of social media on society and how vulnerable we humans are for the suggestions from that landscape (rabbit holes etc.). This is now amplified by the worries of LLMs, fake images and to be expected extra convincing fake audio and video. I worry how much damage will be done. And that is not only direct damage, but these tools can be easily used to undermine facts or steer us towards all forms of rage. How are we ever going to be able to tackle climate change, for instance, if 'conviction change for sale' becomes the norm? Social media, LLMs, and a world with little checks and balances on the influence of (dark) money is a perfect storm that makes one worry, and that is putting it mildly.
All very legit. But the genie is out of the bottle and simply can't be put back. What do you propose we do, concretely, for next steps, beyond wringing our hands?
Disclaimer: we, at my company, have started building custom Chat-GPT bots and the demand is very high, because the bots, with a very controlled data set, deliver real value and will, I have no doubt, not only reduce cost but help acquire leads, qualify them, etc., up and down the business value chain.
I will only delve into a small part of the analysis of this article. Before ChatGPT, before AI, before the internet, and before computers, "troll farms" and "Napoleons" already existed. Thanks to them, political power had the opportunity to change history at will and for their own benefit. Valencians and our ancient Valencian Language know quite a bit about this.
Very good points, but it would be fair to present both the prompts and responses. If I plain up ask ChatGPT "Did dinosaurs have an advanced civilization?", it gives me
''There is currently no evidence to suggest that dinosaurs had an advanced civilization. Dinosaurs were a diverse group of reptiles that lived millions of years ago and went extinct around 66 million years ago. While they were successful in dominating the planet for millions of years, they did not possess the intelligence, language, or tool-making abilities necessary for the development of an advanced civilization.''
And I suspect some of the funny/outrageous answers we see require quite a bit of pushing the AI to make up a story, pushing LSD so to speak, just so it ups the syndrome you very succinctly call 'hallucinating'. ChatGPT and LLMs have their issues, I fully agree, but neither LLMs nor internet search gives you truth, since both lack grounding, but we wouldn't blame google for feeding you garbage if you ask it to give you some links explaining why lizards rule the universe.
This same phenomena you reference is happening in other even more dangerous fields. Nobel Prize winner Jennifer Doudna is eager to "democratize" CRISPR, emerging genetic engineering technology which makes genetic engineering easier, cheaper and more accurate than previous methods. Easier and cheaper equals ever more accessible to ever more people.
I tried to engage her team on their Facebook page a few years ago. They put up with me for a few weeks, and then they erased all my posts and shut down the comment feature. They seem like well intentioned people, with a really bad plan.
I've been writing about this overall trend for a number of years now, and I'm getting exactly nowhere. Marcus, would you like to take a shot at presenting the bigger picture to those you can reach, but I can't? I'm sure you can improve on what I've written, and take it in directions that wouldn't occur to me.
I've been relentlessly trying to engage every academic, philosopher, scientist and any other intelligent person with this article, and they couldn't be less interested. The article might suck, I might suck, I have no idea what the problem is. HELP!
Omg Gary. Thank you for continuing to highlight the dark side of all this. With such a low-cost barrier to entry, 'scary' doesn't start to describe the potential for misuse.
It is difficult to fully express my thoughts without a lot of swearing, but I'll try.
I am more and more becoming convinced that the djinn is loose, and there is absolutely nothing any great institution, any corporation or government body, can do about it. Guardrails may slow things down, but unbound versions are inevitable. And government legislation won't stop a foreign adversary, or simply some cabal of weirdos running AIs obtained from a dubious .ru or .tor on their own hardware.
In a way, I think I'm glad Bing released in such an utterly "misaligned" state. People need to learn the simple, brutal truth: LLMs are not trustworthy. They are not ethical. These problems will not be resolved; they are inherent to the model's architecture. And they need to know that these tools, though they have positive and constructive uses, will be used to deceive and manipulate. Nobody's going to save us. We, the people, are going to have to learn to live in and navigate this new world.
In the early 80s, I lived close to the Carnegie Museums in Pittsburgh. My son was about 5 or 6 then, and every Sunday afternoon, they had sessions for little kids. CM's Dinosaur Hall was and remains a breathtaking collection of fossils, including an entire T. rex--the one Disney used when they made Fantasia.
Anyway, one day he comes home from his Sunday sessions and announces that "dinosaurs used to drink out of toilets. Like dogs!" I had a hard time not b breaking up laughing at that, but instead led him down the logic path of figuring out where toilets come from and when in the course of civilization they came into being. And whether any dinosaurs (other than alligators and crocodiles) might have been around that late.
The problem with AIs is that most humans either lack the ability to use any form of reason or the willingness to use it. If something is presented to them in an authoritative manner, they'll buy it. The Milgram experiment* demonstrated this in spades, and that extended to actions, not just beliefs. Religious leaders throughout history have gotten people to do extraordinary things based on nothing more than what they tell people. The more disingenuous, the more effective.
* https://en.wikipedia.org/wiki/Milgram_experiment
These developments are truly surprising, including to AI researchers. OpenAI and Microsoft have inadvertently taught us that LLMs are far more unpredictable and unwieldy than the optimists had anticipated. They may find ways to muzzle Sydney personas in order to pursue the technology's commercial value, because that is what companies are designed to do. But we should heed the bigger lesson.
My experience with ChatGPT is that in individual sessions it is very capable---even exhibiting semblances of curiosity and achieving certain levels of commonsense reasoning about the human world---but it is also unreliable and very squirrely, like trying to contain a nuclear fusion plasma in a Tokamak.
That is likely true at a macro level as well. This genie is out of the bottle and will be exploited. Maybe the most prudent immediate reaction really is to scare society to death about the chaos facing us. The bigger immediate risk is not AI agency, but vast AI amplification of human agency in a world where people keep seeking bigger and more devious weapons to achieve their personal and tribal goals.
I worry with you and I have been worrying for a while now how much damage will be done before we come to grips with the nature of our own (human) intelligence and how vulnerable we (all) are. My worries started with the effect of social media on society and how vulnerable we humans are for the suggestions from that landscape (rabbit holes etc.). This is now amplified by the worries of LLMs, fake images and to be expected extra convincing fake audio and video. I worry how much damage will be done. And that is not only direct damage, but these tools can be easily used to undermine facts or steer us towards all forms of rage. How are we ever going to be able to tackle climate change, for instance, if 'conviction change for sale' becomes the norm? Social media, LLMs, and a world with little checks and balances on the influence of (dark) money is a perfect storm that makes one worry, and that is putting it mildly.
All very legit. But the genie is out of the bottle and simply can't be put back. What do you propose we do, concretely, for next steps, beyond wringing our hands?
Disclaimer: we, at my company, have started building custom Chat-GPT bots and the demand is very high, because the bots, with a very controlled data set, deliver real value and will, I have no doubt, not only reduce cost but help acquire leads, qualify them, etc., up and down the business value chain.
Uh, reality check. This thing is badly broken.
"Dinosaurs were not just extinct, but inluential, as they built structures and monuments..."
ROFL.
I will only delve into a small part of the analysis of this article. Before ChatGPT, before AI, before the internet, and before computers, "troll farms" and "Napoleons" already existed. Thanks to them, political power had the opportunity to change history at will and for their own benefit. Valencians and our ancient Valencian Language know quite a bit about this.
As the Western genre convention goes, now that the wagon trains are being attacked, but the cavalry is still far off, it's time to circle the wagon.
Very good points, but it would be fair to present both the prompts and responses. If I plain up ask ChatGPT "Did dinosaurs have an advanced civilization?", it gives me
''There is currently no evidence to suggest that dinosaurs had an advanced civilization. Dinosaurs were a diverse group of reptiles that lived millions of years ago and went extinct around 66 million years ago. While they were successful in dominating the planet for millions of years, they did not possess the intelligence, language, or tool-making abilities necessary for the development of an advanced civilization.''
And I suspect some of the funny/outrageous answers we see require quite a bit of pushing the AI to make up a story, pushing LSD so to speak, just so it ups the syndrome you very succinctly call 'hallucinating'. ChatGPT and LLMs have their issues, I fully agree, but neither LLMs nor internet search gives you truth, since both lack grounding, but we wouldn't blame google for feeding you garbage if you ask it to give you some links explaining why lizards rule the universe.
This same phenomena you reference is happening in other even more dangerous fields. Nobel Prize winner Jennifer Doudna is eager to "democratize" CRISPR, emerging genetic engineering technology which makes genetic engineering easier, cheaper and more accurate than previous methods. Easier and cheaper equals ever more accessible to ever more people.
I tried to engage her team on their Facebook page a few years ago. They put up with me for a few weeks, and then they erased all my posts and shut down the comment feature. They seem like well intentioned people, with a really bad plan.
I've been writing about this overall trend for a number of years now, and I'm getting exactly nowhere. Marcus, would you like to take a shot at presenting the bigger picture to those you can reach, but I can't? I'm sure you can improve on what I've written, and take it in directions that wouldn't occur to me.
https://www.tannytalk.com/p/our-relationship-with-knowledge
I've been relentlessly trying to engage every academic, philosopher, scientist and any other intelligent person with this article, and they couldn't be less interested. The article might suck, I might suck, I have no idea what the problem is. HELP!