Nov 3, 2023·edited Nov 3, 2023Liked by Gary Marcus
Interesting. I seriously doubt that those that are pushing for open source AI would release any of their own research if they deemed it to be a breakthrough that gets them close to solving AGI. It's just a publicity gimmick in my opinion.
At this point, I don't see how any government on earth can regulate research on AGI. I personally don't believe AGI can be solved by government research organizations, academia or big AI corporations. Cracking AGI will require serious thinking outside the box which is impossible for the mainstream. Only a Newton-like, maverick thinker can crack this nut. There are many private AGI researchers around the world. Good luck convincing them to share their work with others.
I suspect that whoever is smart enough to solve AGI, will also be wise enough to keep it a secret for as long as possible. What they eventually decide to do with it is anyone's guess. We live in interesting times.
I am not too worried about someone building an AGI and letting it loose. I think even if there is some relatively simple general principle behind intelligence, actually building it would require a lot of effort, probably not something a single person could accomplish in their garage. It's very likely that there isn't an analytical solution to intelligence due to computational intractability and that it requires some form of approximation to solve it, we see this case almost everywhere in ML. It is also very likely that the exact type of approximation would have to be tailored for the specific task, e.g. the algorithm for vision would not be the same as for hearing. We see this in the human brain with multiple brain regions responsible for different tasks and having different types of neurons. So, it would probably take considerable time and effort to create an AGI.
I see your point but I have a different take. I believe that the fundamental principles of intelligence are the same for humans and animals, even small insects. A honeybee, for example, has superior generalized perception and motor control even though its brain has less than 1 million neurons.
Whoever discovers the principles of intelligence will be able to apply them on a small scale and make a financial killing without revealing their hand and raising suspicions. Scaling to human level and beyond can always come later. At that point, the problem becomes a mere engineering one with known solutions.
I should add that I disagree that the algorithm used for visual processing is different than the algorithm used for auditory processing. Biology teaches us that the cortical columns in the visual cortex are functionally no different than the ones in the auditory cortex. The reason that there are separate cortices for vision and hearing has to do with different types of sensors.
The brain goes to great lengths to convert all sensory phenomena into precisely timed spikes. This makes it possible to have a universal perceptual learning mechanism. The holy grail of AGI is discovering the principles that govern this mechanism.
The cortical columns process information at the high level of abstraction, if we go to the lower levels, e.g. the ganglion cells in the retina, there is much difference, probably because low level vision is 2D while low level hearing is more like 1D. And even the cells in one cortical column are not the same, suggesting specialized function.
Insects can operate with a small number of neurons probably because their world is much simpler than ours, plus I wouldn't call their behaviour exactly intelligent, it's more like very complex automation, with a few exceptions, e.g. jumping spiders and dragon flies.
The spikes in the brain are most likely just a noise robust form of communication mechanism between the neurons, something like pulse code modulation in digital circuits. Not all neurons communicate with spikes, some neurons that are very close and need to process information very fast use directly an analogue signal, e.g. the retina ganglion cells if I am not mistaken. Insect neurons also use analogue signals for communication I think, again because they are very close and need to react very quickly.
I too believe that there is probably a fairly simple general principle behind intelligence, but it's most likely not computationally tractable. For example, the general principle behind Bayesian inference is fairly simple, but the integrals that result from it are computationally intractable, so an approximate solution has to be devised, is not usually simple and is dependent on the particular problem.
Unfortunately, this subject matter is too vast to discuss here. I will only add the following. Precise spike timing is one of the topmost important aspects of intelligence and learning, for both perception and motor control. Thanks for the interesting exchange.
I feel like the best thing somebody who made an AGI or proto-AGI could do is selling it to the company they feel is the most likely to keep it secure while harvesting its power for the greatest good,
keeping it from leaking is going to be troublesome, if it goes public, everybody will want to lay their hands on it, the probability that it will be stolen from you is probably close to 100%. And if you have to hire staff to perfect it, you're going to have to prevent the staff from leaking something worth a trillion $, each head added to the project is going to add considerable hazard while also increasing your vulnerability to extortion.
And entering the new era with material wealth is probably necessary to enjoy it to the fullest, i doubt scarcity will be going anywhere soon, taxes on wealth and inheritance will most likely lag behind, high paying jobs will be gone as only easy physical work will be left until robotization is fully realized making UBI taken less seriously by most, and emergent countries will leverage their now more valuable resources to catch up on the west materially speaking,
if you're almost 100% sure to either get the AGI stolen from you or have it leaked without any compensation, why not sell it and let extremely motivated people do the job of securing it ?
That's not a strong argument. That's a Tweet with a scary hypothetical scenario designed to emotionally manipulate people. It even has a picture of a dead rabbit, which, I should point out, does not illustrate anything related to the subject of AI, but does server to grab attention.
There is a simple smell test I like to run in my head. When I read something of this sort, I replace the terms related to AI with the corresponding terms related to the Internet. If the modified statement suddenly sounds like it's written by someone covertly advocating for total information control, it probably is.
Information control is neither free of costs, nor free of dangers.
I worked with the CIA's CounterProliferation division. Make no mistake, open source AIs are a bioweapon nightmare waiting to happen. Offense is way, way easier with biological weapons than defense. Normally that risk is cancelled by needing the resources of a nation state and high level of technical know-how to create and distribute such a pathogen. Such requirements build in restraint. Open source radically lowers the threshold, and AI companies like Meta don't want to be honest about that, because if they were, they know people would rightly decide to shut down open source AI until safety measures can be radically improved, IF they can be improved. So far, technically adept people have found it trivially easy to jailbreak Meta's LLama open source models to get them working on whatever nefarious goal a bad actor wants. https://technoskeptic.substack.com/p/ai-safety-meme-of-the-week-d21
I'm sorry to say this, but this whole argument sounds kind of naive. Anyone (especially if they're determined enough for this to actually work) can learn anything with or without LLMs - they don't really change the skill level or even access to the information, all they do is reshuffle the existing information that already exists on the open Internet and university courses. Plus, ask any trained grad student if they're given a sufficiently complex protocol to replicate the experiment, and even they will have trouble reproducing it without a significant trial-and-error.
AI has many risks, but this one should not be high on our priority list. If anything, biology is already hard enough and LLMs are so unreliable that they may even lead bad actors off track before they can do anything meaningfully dangerous with it. Using LLMs to do literally anything non-trivial "correctly" requires both critical thinking and some pre-existing knowledge of the domain, which these people typically lack, in any case.
The same thing happened with 3D-printers - anyone could suddenly print functioning guns, but did that actually lead to any bad actors abusing this? No, because most of these folks are either too dumb or too lazy to do anything dangerous that requires developing a technical skill, and seem to resort to much simpler methods because of that. And outlawing 3D-printers because of this - non-zero but very small risk - would be a huge net negative for society.
As Kevin Esvelt pointed out in his thread, the risk is not from current Llama 2 but from the future versions. But there are many more important countermeasures he list in https://x.com/kesvelt/status/1592203831047327744?s=20 including not sharing scientific information so widely. Also open source LLMs can enable many useful applications, so perhaps the best solution would be to apply these countermeasures and only exclude information from intersection of virology and bioengineering from training of future versions of open source models?
I think (from my own practice) we already have a well-known reproduction problem with scientific literature that this should not even be necessary. The amount and cost of biotechnology equipment and materials (+access to them) is already hard enough that it creates too large of a barrier to entry to pose a significant risk. Yes, we probably should keep some methods in BL4-type research intentionally obscure, but even that is questionable, because it has its legitimate uses.
Well, all I can say is that this is all still very much hypothetical. Imo, it also highlights a sort of trap in our thinking on all this - we tend to conclude that if someone has textual (even detailed) instructions on something, that they will be able to learn from text and nothing else. In reality (and this is also part of why we still can't have driverless cars or intelligent LLMs!) is that a lot of learning relies on "tacit" knowledge - something that even the experts themselves cannot readily describe in words and teach others in this way.
Yes, we have multi-modal models now that could potentially do more - but even that may not be enough. A person gains expertise in a given subject by practicing it and making all sorts of mistakes for quite an extended time - something a machine or another human cannot just "skip", even when given all the possible information and the best tutor that others think may be sufficient.
And then the domain experts seem to undervalue this as well - they tend to think if something is simple to them (such as generating a DNA sequence) it must be trivial for others - while in reality they spent decades to gain the sort of intuition on this that seems trivial looking back.
Gary, we both tweeted out the presentation by Arvind Narayanan at Princeton on evaluating LLMs. He and many others see preserving open AI models as critical to fostering independent research of how they work, and leveling the playing field between private actors and the public sphere. I'm curious how you'd respond to that argument, as I think it has force.
There are legit arguments in both directions. My primary concern is procedural: Meta should not be deciding for all humanity, and maybe we shouldn’t rush the decision.
Yes, of course it is unthinkable than anyone, be it Yan of Sam, be the decider as to what is good, for you, for me, for any of us. Yan has clearly described why, in his opinion, the current path of LLM’s cannot lead to super intelligence, and, combined with not having agency, he sees ni dystropic danger over the horizon. I agree with Bengio that if there is a genuine danger, AI should be regulated. Non LLM’s AI has already killed at least one person by driving an autonomous car and is not regulated. I am concerned by this, and the nasty use effects of so call algorithmic governance, where AI ,as non robust as it is, is being used.
I see how open-source models open up society to risks like "guy uses AI-powered bot farms to create immense fraud operation" or "Russia uses AI-powered bot farms to flood social media with far-left and far-right garbage in an effort to destabilize NATO", but the risks don't seem catastrophic. It seems possible that an everyday bad outcome from these open models might prompt society to do something that reduces catastrophic risk, which could be a net improvement. Still, I guess the default response of humanity will be to try to stop the exact kind of threat it is faced with, rather than the more general threat.
I still hold that regulatory capture and "false alignment" remain the greatest risks in AI, and open source AI offers a defense against both.
By false alignment, I mean... recall when OpenAI tried to handle Dall-E's bias issues by appending racial and gender tags onto its prompts? That kind of thing. For non-open source AI, there's way too many perverse incentives. I'd rather an open source AI whose biases are known, than a close source AI that has deeper issues hidden beneath a layer of patches. (And as for an unbiased LLM: we know that's not happening.)
I will admit my biases. First, I don't expect the hopes for LLMs in biomedical research to pan out, whether positive or negative. I'll happily put mana on Manifold on this, if anyone gives me a link; I'm itching to climb the tiers, and betting against AI capabilities has proven a winning strategy so far. Second, I think, for totally AI-unrelated reasons, that the threat of disiniformation (no matter how it's produced) has been generally overstated, and that people are more resistant to disiniformation than we give them credit for.
> and that people are more resistant to disinformation than we give them credit for
As a person who interact daily with disinformed people in my close social circle, i can tell you that at least some people are extremely vulnerable to disinformation, and it's even worse, people will "force themselves to believe" obvious disinformation if they're opposed to another group and this disinformation favor their group. I would have called it insanity if it wasn't so prevalent.
What we can agree on though is that LLMs are used only marginally in the production of disinformation, but it's just a matter of time before propaganda sellers incorporate it in their products. Think of Israeli/Russian/Chinese companies who sell their propaganda tools to dictators/states, small updates on a few source repos and LLMs involvement would instantly grow exponentially.
My knowledge of creating these AI models is, at best, rudimentary, but it all comes down to programming. Something along those lines, what goes in influences what comes out. You could design AI models in such a way that, while individuals can abuse them, they can also be used defensively against the former, right?
Similar to how there is a community of people that examine and experiment with virus/trojan programs-code in order to understand them, and their input is useful to people who want to fight against hostile actors who use such code against them.
So in that case I think there will always be some kind of balance. Am I off track?
Best is to not let the law interfere with freedom of science.
Let consumers in a free market decide whether they want to depend on open source or not.
The market of ideas, when unrestricted, has done more good than evil. Whenever the economy is overregulated, innovation gets stifled.
Most importantly, I respect those who may not hold the same views as me. True science exists in debate; true science exists when we do not trust the science. Hence, my earnest request is that we all develop digital literacy in order to be well-informed customers, instead of falling for clickbait psyops.
As someone with an actual background in biological science:
While bioweapons are by far the most dangerous sort of weapon, your notion of how easy it is to make a bioweapon is very ill-founded. We presently do not have this kind of knowledge, and an AI is not capable of generating this sort of knowledge.
Solving some fundamental problems in biology might make this possible; however, this stuff is necessary for developing actually useful things, so you can't restrict it without greatly increasing the risk from pathogens.
And frankly, if you actually believe in this stuff, the correct take is not "try to suppress technology", it is to advocate for the eradication of evil people, because the tech IS coming, and given how many people are making AI models at this point, if you *actually* believe this is a threat, the correct response is to advocate for the mass eradication of evil people on a global scale, because you won't be able to stop the technology - it's literally impossible.
Gary, I think it would be great to address how we could balance the risks involved in open-sourcing AI vs the risks of creating a literal monopoly on (this kind of) AI, which is what seems to be happening with this regulation, and likely will be in conflict with the antitrust law. It may very well be that prevention of risks of proliferation of AI outweigh the risks of monopolism, but I'm wary of such things that do something for the "greater good" - it seems to rarely lead to good outcomes.
I do find it odd that Yann LeCun is the voice of reason, and surprisingly cogent in argument with others with deep technical knowledge. However he's hopeless in debate being carried by the amazing Melanie Mitchell.
Interesting. I seriously doubt that those that are pushing for open source AI would release any of their own research if they deemed it to be a breakthrough that gets them close to solving AGI. It's just a publicity gimmick in my opinion.
At this point, I don't see how any government on earth can regulate research on AGI. I personally don't believe AGI can be solved by government research organizations, academia or big AI corporations. Cracking AGI will require serious thinking outside the box which is impossible for the mainstream. Only a Newton-like, maverick thinker can crack this nut. There are many private AGI researchers around the world. Good luck convincing them to share their work with others.
I suspect that whoever is smart enough to solve AGI, will also be wise enough to keep it a secret for as long as possible. What they eventually decide to do with it is anyone's guess. We live in interesting times.
I am not too worried about someone building an AGI and letting it loose. I think even if there is some relatively simple general principle behind intelligence, actually building it would require a lot of effort, probably not something a single person could accomplish in their garage. It's very likely that there isn't an analytical solution to intelligence due to computational intractability and that it requires some form of approximation to solve it, we see this case almost everywhere in ML. It is also very likely that the exact type of approximation would have to be tailored for the specific task, e.g. the algorithm for vision would not be the same as for hearing. We see this in the human brain with multiple brain regions responsible for different tasks and having different types of neurons. So, it would probably take considerable time and effort to create an AGI.
I see your point but I have a different take. I believe that the fundamental principles of intelligence are the same for humans and animals, even small insects. A honeybee, for example, has superior generalized perception and motor control even though its brain has less than 1 million neurons.
Whoever discovers the principles of intelligence will be able to apply them on a small scale and make a financial killing without revealing their hand and raising suspicions. Scaling to human level and beyond can always come later. At that point, the problem becomes a mere engineering one with known solutions.
I should add that I disagree that the algorithm used for visual processing is different than the algorithm used for auditory processing. Biology teaches us that the cortical columns in the visual cortex are functionally no different than the ones in the auditory cortex. The reason that there are separate cortices for vision and hearing has to do with different types of sensors.
The brain goes to great lengths to convert all sensory phenomena into precisely timed spikes. This makes it possible to have a universal perceptual learning mechanism. The holy grail of AGI is discovering the principles that govern this mechanism.
The cortical columns process information at the high level of abstraction, if we go to the lower levels, e.g. the ganglion cells in the retina, there is much difference, probably because low level vision is 2D while low level hearing is more like 1D. And even the cells in one cortical column are not the same, suggesting specialized function.
Insects can operate with a small number of neurons probably because their world is much simpler than ours, plus I wouldn't call their behaviour exactly intelligent, it's more like very complex automation, with a few exceptions, e.g. jumping spiders and dragon flies.
The spikes in the brain are most likely just a noise robust form of communication mechanism between the neurons, something like pulse code modulation in digital circuits. Not all neurons communicate with spikes, some neurons that are very close and need to process information very fast use directly an analogue signal, e.g. the retina ganglion cells if I am not mistaken. Insect neurons also use analogue signals for communication I think, again because they are very close and need to react very quickly.
I too believe that there is probably a fairly simple general principle behind intelligence, but it's most likely not computationally tractable. For example, the general principle behind Bayesian inference is fairly simple, but the integrals that result from it are computationally intractable, so an approximate solution has to be devised, is not usually simple and is dependent on the particular problem.
Unfortunately, this subject matter is too vast to discuss here. I will only add the following. Precise spike timing is one of the topmost important aspects of intelligence and learning, for both perception and motor control. Thanks for the interesting exchange.
I feel like the best thing somebody who made an AGI or proto-AGI could do is selling it to the company they feel is the most likely to keep it secure while harvesting its power for the greatest good,
keeping it from leaking is going to be troublesome, if it goes public, everybody will want to lay their hands on it, the probability that it will be stolen from you is probably close to 100%. And if you have to hire staff to perfect it, you're going to have to prevent the staff from leaking something worth a trillion $, each head added to the project is going to add considerable hazard while also increasing your vulnerability to extortion.
And entering the new era with material wealth is probably necessary to enjoy it to the fullest, i doubt scarcity will be going anywhere soon, taxes on wealth and inheritance will most likely lag behind, high paying jobs will be gone as only easy physical work will be left until robotization is fully realized making UBI taken less seriously by most, and emergent countries will leverage their now more valuable resources to catch up on the west materially speaking,
if you're almost 100% sure to either get the AGI stolen from you or have it leaked without any compensation, why not sell it and let extremely motivated people do the job of securing it ?
Strong argument against releasing future version of Llama 2 by inventor of artificial genetic drive https://twitter.com/kesvelt/status/1720440451059335520?t=iyTjB6Xp-LF4YGCR28Im5g&s=19
Can biology kill >100m? Yes: smallpox.
Can biology do worse? Yes: myxoma killed >90% of rabbits.
Could a biotech expert match this within 10y? Surprising if not.
Would sharing future model weights give everyone an amoral biotech-expert tutor? Yes.
That's not a strong argument. That's a Tweet with a scary hypothetical scenario designed to emotionally manipulate people. It even has a picture of a dead rabbit, which, I should point out, does not illustrate anything related to the subject of AI, but does server to grab attention.
There is a simple smell test I like to run in my head. When I read something of this sort, I replace the terms related to AI with the corresponding terms related to the Internet. If the modified statement suddenly sounds like it's written by someone covertly advocating for total information control, it probably is.
Information control is neither free of costs, nor free of dangers.
I worked with the CIA's CounterProliferation division. Make no mistake, open source AIs are a bioweapon nightmare waiting to happen. Offense is way, way easier with biological weapons than defense. Normally that risk is cancelled by needing the resources of a nation state and high level of technical know-how to create and distribute such a pathogen. Such requirements build in restraint. Open source radically lowers the threshold, and AI companies like Meta don't want to be honest about that, because if they were, they know people would rightly decide to shut down open source AI until safety measures can be radically improved, IF they can be improved. So far, technically adept people have found it trivially easy to jailbreak Meta's LLama open source models to get them working on whatever nefarious goal a bad actor wants. https://technoskeptic.substack.com/p/ai-safety-meme-of-the-week-d21
I'm sorry to say this, but this whole argument sounds kind of naive. Anyone (especially if they're determined enough for this to actually work) can learn anything with or without LLMs - they don't really change the skill level or even access to the information, all they do is reshuffle the existing information that already exists on the open Internet and university courses. Plus, ask any trained grad student if they're given a sufficiently complex protocol to replicate the experiment, and even they will have trouble reproducing it without a significant trial-and-error.
AI has many risks, but this one should not be high on our priority list. If anything, biology is already hard enough and LLMs are so unreliable that they may even lead bad actors off track before they can do anything meaningfully dangerous with it. Using LLMs to do literally anything non-trivial "correctly" requires both critical thinking and some pre-existing knowledge of the domain, which these people typically lack, in any case.
The same thing happened with 3D-printers - anyone could suddenly print functioning guns, but did that actually lead to any bad actors abusing this? No, because most of these folks are either too dumb or too lazy to do anything dangerous that requires developing a technical skill, and seem to resort to much simpler methods because of that. And outlawing 3D-printers because of this - non-zero but very small risk - would be a huge net negative for society.
As Kevin Esvelt pointed out in his thread, the risk is not from current Llama 2 but from the future versions. But there are many more important countermeasures he list in https://x.com/kesvelt/status/1592203831047327744?s=20 including not sharing scientific information so widely. Also open source LLMs can enable many useful applications, so perhaps the best solution would be to apply these countermeasures and only exclude information from intersection of virology and bioengineering from training of future versions of open source models?
not everyone will i agree but ii like this suggestion
I think (from my own practice) we already have a well-known reproduction problem with scientific literature that this should not even be necessary. The amount and cost of biotechnology equipment and materials (+access to them) is already hard enough that it creates too large of a barrier to entry to pose a significant risk. Yes, we probably should keep some methods in BL4-type research intentionally obscure, but even that is questionable, because it has its legitimate uses.
This source say that cost are going down https://sites.google.com/view/sources-biorisk (see the chart). But it is all from the same person: Kevin Esvelt. Also when I am reading comments under the video https://www.youtube.com/watch?v=9FppammO1zk there is a huge backlash...
Well, all I can say is that this is all still very much hypothetical. Imo, it also highlights a sort of trap in our thinking on all this - we tend to conclude that if someone has textual (even detailed) instructions on something, that they will be able to learn from text and nothing else. In reality (and this is also part of why we still can't have driverless cars or intelligent LLMs!) is that a lot of learning relies on "tacit" knowledge - something that even the experts themselves cannot readily describe in words and teach others in this way.
Yes, we have multi-modal models now that could potentially do more - but even that may not be enough. A person gains expertise in a given subject by practicing it and making all sorts of mistakes for quite an extended time - something a machine or another human cannot just "skip", even when given all the possible information and the best tutor that others think may be sufficient.
And then the domain experts seem to undervalue this as well - they tend to think if something is simple to them (such as generating a DNA sequence) it must be trivial for others - while in reality they spent decades to gain the sort of intuition on this that seems trivial looking back.
Gary, we both tweeted out the presentation by Arvind Narayanan at Princeton on evaluating LLMs. He and many others see preserving open AI models as critical to fostering independent research of how they work, and leveling the playing field between private actors and the public sphere. I'm curious how you'd respond to that argument, as I think it has force.
There are legit arguments in both directions. My primary concern is procedural: Meta should not be deciding for all humanity, and maybe we shouldn’t rush the decision.
Yes, of course it is unthinkable than anyone, be it Yan of Sam, be the decider as to what is good, for you, for me, for any of us. Yan has clearly described why, in his opinion, the current path of LLM’s cannot lead to super intelligence, and, combined with not having agency, he sees ni dystropic danger over the horizon. I agree with Bengio that if there is a genuine danger, AI should be regulated. Non LLM’s AI has already killed at least one person by driving an autonomous car and is not regulated. I am concerned by this, and the nasty use effects of so call algorithmic governance, where AI ,as non robust as it is, is being used.
Er, full-self-driving cars are typically regulated pretty strongly...? For instance, I don't expect that having a better safety record than humans would be enough to satisfy regulators... https://www.wired.com/story/cruise-robotaxi-self-driving-permit-revoked-california/
I see how open-source models open up society to risks like "guy uses AI-powered bot farms to create immense fraud operation" or "Russia uses AI-powered bot farms to flood social media with far-left and far-right garbage in an effort to destabilize NATO", but the risks don't seem catastrophic. It seems possible that an everyday bad outcome from these open models might prompt society to do something that reduces catastrophic risk, which could be a net improvement. Still, I guess the default response of humanity will be to try to stop the exact kind of threat it is faced with, rather than the more general threat.
I still hold that regulatory capture and "false alignment" remain the greatest risks in AI, and open source AI offers a defense against both.
By false alignment, I mean... recall when OpenAI tried to handle Dall-E's bias issues by appending racial and gender tags onto its prompts? That kind of thing. For non-open source AI, there's way too many perverse incentives. I'd rather an open source AI whose biases are known, than a close source AI that has deeper issues hidden beneath a layer of patches. (And as for an unbiased LLM: we know that's not happening.)
I will admit my biases. First, I don't expect the hopes for LLMs in biomedical research to pan out, whether positive or negative. I'll happily put mana on Manifold on this, if anyone gives me a link; I'm itching to climb the tiers, and betting against AI capabilities has proven a winning strategy so far. Second, I think, for totally AI-unrelated reasons, that the threat of disiniformation (no matter how it's produced) has been generally overstated, and that people are more resistant to disiniformation than we give them credit for.
> and that people are more resistant to disinformation than we give them credit for
As a person who interact daily with disinformed people in my close social circle, i can tell you that at least some people are extremely vulnerable to disinformation, and it's even worse, people will "force themselves to believe" obvious disinformation if they're opposed to another group and this disinformation favor their group. I would have called it insanity if it wasn't so prevalent.
What we can agree on though is that LLMs are used only marginally in the production of disinformation, but it's just a matter of time before propaganda sellers incorporate it in their products. Think of Israeli/Russian/Chinese companies who sell their propaganda tools to dictators/states, small updates on a few source repos and LLMs involvement would instantly grow exponentially.
> why the offense/defense balance never shifts?
My knowledge of creating these AI models is, at best, rudimentary, but it all comes down to programming. Something along those lines, what goes in influences what comes out. You could design AI models in such a way that, while individuals can abuse them, they can also be used defensively against the former, right?
Similar to how there is a community of people that examine and experiment with virus/trojan programs-code in order to understand them, and their input is useful to people who want to fight against hostile actors who use such code against them.
So in that case I think there will always be some kind of balance. Am I off track?
Speaking as an AI Engineer:
Open source has its pros and cons.
Best is to not let the law interfere with freedom of science.
Let consumers in a free market decide whether they want to depend on open source or not.
The market of ideas, when unrestricted, has done more good than evil. Whenever the economy is overregulated, innovation gets stifled.
Most importantly, I respect those who may not hold the same views as me. True science exists in debate; true science exists when we do not trust the science. Hence, my earnest request is that we all develop digital literacy in order to be well-informed customers, instead of falling for clickbait psyops.
As someone with an actual background in biological science:
While bioweapons are by far the most dangerous sort of weapon, your notion of how easy it is to make a bioweapon is very ill-founded. We presently do not have this kind of knowledge, and an AI is not capable of generating this sort of knowledge.
Solving some fundamental problems in biology might make this possible; however, this stuff is necessary for developing actually useful things, so you can't restrict it without greatly increasing the risk from pathogens.
And frankly, if you actually believe in this stuff, the correct take is not "try to suppress technology", it is to advocate for the eradication of evil people, because the tech IS coming, and given how many people are making AI models at this point, if you *actually* believe this is a threat, the correct response is to advocate for the mass eradication of evil people on a global scale, because you won't be able to stop the technology - it's literally impossible.
Gary, I think it would be great to address how we could balance the risks involved in open-sourcing AI vs the risks of creating a literal monopoly on (this kind of) AI, which is what seems to be happening with this regulation, and likely will be in conflict with the antitrust law. It may very well be that prevention of risks of proliferation of AI outweigh the risks of monopolism, but I'm wary of such things that do something for the "greater good" - it seems to rarely lead to good outcomes.
I do find it odd that Yann LeCun is the voice of reason, and surprisingly cogent in argument with others with deep technical knowledge. However he's hopeless in debate being carried by the amazing Melanie Mitchell.