Where we are right now on open source and potential AI risk.
It’s not pretty
1. Nobody knows for SURE that there are no serious possible consequences of open source AI, e.g in terms of bioweapons, misinformation generation etc, as Séb Krier at DeepMind nicely highlights above.
2. A giant company that is behind in the AI sweepstakes (at least with respect to public profile) is aggressively pushing open source AI, apparently as a way to raise its profile
3. They have done this as quickly as possible, apparently without any external oversight whatsoever, without waiting for anything like formal results or scientific consensus.
4. There is a nonzero chance that they have put us all at massive risk, with zero oversight.
Not great. We really ought be a bit more patient about comparing risks and benefits.
The fact that a single company can unilaterally make this decision for all of humanity is terrifying.
Gary Marcus sincerely hopes this all turns out ok.
Interesting. I seriously doubt that those that are pushing for open source AI would release any of their own research if they deemed it to be a breakthrough that gets them close to solving AGI. It's just a publicity gimmick in my opinion.
At this point, I don't see how any government on earth can regulate research on AGI. I personally don't believe AGI can be solved by government research organizations, academia or big AI corporations. Cracking AGI will require serious thinking outside the box which is impossible for the mainstream. Only a Newton-like, maverick thinker can crack this nut. There are many private AGI researchers around the world. Good luck convincing them to share their work with others.
I suspect that whoever is smart enough to solve AGI, will also be wise enough to keep it a secret for as long as possible. What they eventually decide to do with it is anyone's guess. We live in interesting times.
Strong argument against releasing future version of Llama 2 by inventor of artificial genetic drive https://twitter.com/kesvelt/status/1720440451059335520?t=iyTjB6Xp-LF4YGCR28Im5g&s=19
Can biology kill >100m? Yes: smallpox.
Can biology do worse? Yes: myxoma killed >90% of rabbits.
Could a biotech expert match this within 10y? Surprising if not.
Would sharing future model weights give everyone an amoral biotech-expert tutor? Yes.