As recently as November 2023, OpenAI promised in their filing as a nonprofit exempt from income tax to make AI that that “benefits humanity … unconstrained by a need to generate financial return”.
I haven't noticed much in the way of ethical behaviour from Google; rather the reverse if anything. Their AI project may currently be less bad than OpenAI, but I expect that it will regress to the Google norm in time.
Open source development is DEFINITELY NOT the way to safety. Its bad enough having to corral corporations without potentially handing extremely powerful technologies to rival governments, terrorists and others like accelerationists who explicitly seek the end of humanity.
Anthropic has at least produced good research, which can be used to prevent or lower the risk of AI takeover. In fact, in terms of capabilities versus safety, they've delivered safety more than anyone else.
Shooting yourself in the face to spite your nose is no way to survive a future, especially with all of humanity on the line with us. We also don't open source nuclear weapons, and biological weapons for reason.
If you have low expectations you're hard to disappoint, so I get it - the world is a rough place. At the same time I think reverting to cynicism about the company as a whole is unfair to the AI team who are genuinely doing good things.
Anything can of course still happen in the long run but I believe good efforts deserve recognition and praise, otherwise you're contributing to a world where eventually nobody sees the point in doing any good.
I'm retired from working in digital technology, and have consequently seen far too much of how the sausage tends to be made.
In particular, I've seen a metric ton of hypocrisy - claiming one thing while doing another. This usually comes from the top down, and some of the youngest software engineers believe and are inspired by the idealistic hype, but others understand they are supposed to "say what I say, but do what I do," with the two wildly inconsistent.
Obviously I haven't worked for every single tech company, and in particular have not personally worked for Google. (But I know people who have, and it sounds like a normal big tech company to me.) There are plenty of decent people there, behaving decently except when instructed otherwise, and sometimes even working around their instructions. That's true almost everywhere. But it tends not to be enough to prevent overall bad effects, when incentives skew towards encouraging bad behaviour, particularly when the bad behaviour is at the expense of strangers (such as the general public, future generations, etc.)
I agree but like you said this is mostly the same everywhere.
I still think it is better to have a slogan that says "do no evil" versus "let the world burn".
It will impact the people you attract as you said and therefore your company, whether the board likes it or not.
If you're point is that it is sad that people will be disappointed, face disillusion, feel misled and may end up cynical I agree.
But I don't think changing the slogan to "let the world burn" is a constructive solution even though - once you end up cynical - it may feel more appropriate. But that is also because a part of cynicism is that it means you have given up and might want to see things burn out of spite.
The biggest challenge in life is to stay optimistic and hopeful and strive for the best. You will end up disappointed many times but it can still be the best strategy with the best outcomes overall.
This is also why I think having the same people in power too long is risky. Cynism easily leads to corruption because why should you continue to care of time and time others disappoint you?
This is not to discriminate against old people individually, just to say it's challenging to shed naivety without replacing it with cynicism. To keep trying and striving is exhausting and deserves respect.
I am glad that there are people like you still trying to improve human behaviour. Without those efforts, things would be even worse than they actually are.
Balance is required - history is littered with attempts to create perfection, that turned into particularly nasty oppressive regimes. It's also littered with martyrs, many (most?) to causes that never succeeded.
But it's also full of real improvements, both short term (something was good for a while) and long term (e.g. the very long term trend of reduction of violent death - see Pinker if this is unfamiliar).
Each person gets to judge for themselves in each particular case whether to tilt against those particular windmills/try to right those particular wrongs.
Those choices are partly a matter of personality, partly of their personal situation, and partly of experience. Age is probably also a factor, as you suggest. (I'm not sure whether my current cynicism is due to specific experiences, generational experiences - the 1960s were more optimistic than the 2020s - or personality.)
Yeah I wasn't trying to say you as a person are very cynical - even if you displayed some cynicism.
Sometimes we have to vent or are in a bad mood. Everyone is cynical about things sometimes. I know I've posted some rants here and there in my time. It depends also on what occupies you at any given time.
Maybe that's the good thing, that negativity and cynicism can also weather with time.
FWIW, Substack has the ability to allow users to edit their own replies for some period of time after making them. Marcus has this enabled, you get to it via the horizontal row of three dots on the far right of the row with like, reply and share buttons.
The story with Google and China is complicated and goes back many years. They've flip-flopped a few times now about their approach to working (or not) with the CCP. But there was a time when they enforced the censorship in the 00's, and they were working on a new censored search product as of 2018 strictly for the Chinese market. Background here:
They are hanging themselves via there own words!! Impressive investigation Gary. I tip my hat. Much respect. This I hope decrease the haze that these companies seem to cause people. #beautiful
Yes the public will be convinced this but some will think for themselves. What I call a 'distinct minority'. It is the mindset I focus on. I like your assessment.
Worrying about OpenAI 'winding up first to AGI' kind of suggests they potentially are on a road there. Do you really want to entertain that suggestion?
I don't think Gary has ever said otherwise, or suggested AGI wasn't possible? I'd guess he thinks the mental models of high-ups at OpenAI are wrong though. In the past, Sam Altman has acted in a way consistent with the hypothesis that he thought scaling up GPTs larger and larger would lead directly to AGI, and Gary has always (correctly) said it won't. But there are tons of smart people at OpenAI, and as long as the money keeps rolling in I expect them to eventually make some important technical breakthroughs toward AGI, which they say is their only mission after all.
Mr Marcus, in many of your posts, you mention current efforts as a dead-end on the road to AGI (I agree with you there). But in some other posts like this one, you mention “who gets to AGI first” which seems as if the road actually leads there after all. Where do I misunderstand you please, could you help me clarify?
I think it’s meant rethorical: a company, any company for that matter, that has AGI as its mission should be held at the highest standards and be given maximum scrutiny.
All this fuss about OpenAI is a distraction. This is a quiet period before piracy storm.
Soon we’ll have AI chips in consumer devices and software that will provide user friendly ways for training LLMs for your own personal use.
And then it will be same as Napster days, but instead of MP3s we will download (unofficial) snapshots of Reddit, Quora, StackOverflow, etc. and train your personal AI.
And you will be able to take movie/tv show recordings and train your personal AI to be any actor or celebrity.
Remember old days of TomTom navigator and how you could download spoken directions by celebrities (imitations) despite them never recorded those?
Good point. If I may be optimistic for a moment, this desire to be closer to the celebrity in question points to the powerful social draw that underlies how we respond to celebrity. That makes celebrity (and the artistry that creates it) intrinsically hugely valuable. AI is never going to replace that; "I'm a huge fan of Stockfish" just isn't a thing even if it can defeat Magnus Carlsen every game it plays. People with talent will continue to earn good-to-great livings even if they don't benefit every time someone clicks. As Oliver Wilde quipped: "there is only one thing worse than being talked about, and that is not being talked about".
I don't think we have to worry about OpenAI finding their AGI Holy Grail. They danger is in them successfully convincing others to go sticking AI in all sorts of places it doesn't belong. That's the business model for almost all of the consumer-products industry.
I think since that whole debacle back in Nov, Sam Altman is now in his "I dont trust anyone anymore" (Lex Fridman pod) phase. As if he think he was so naive before believing in ethical governance or something. He has a marvel baddie look about him now he's been burnt......
I think one needs to define what "benefiting humanity" means exactly because as it is it's too loose. As an example, for the Chinese government it might mean better algorithms to detect faces and identify people. For me, the best thing would be for it go back into the hole it crawled from.
I think ScarJo is not a mistake but a concerted effort to lure an Artist into challenging the fact that someone can have a similar voice.
If so it is a devious trap. They made it so it was very believable they did steal her voice. But they also primed us (and her!) to hear her own voice.
It is hard to say whether the strategy works long term but you have to remember that they're not after convincing the already vocal critics that you may be surrounded by.
To the general public that may not share what are essentially niche concerns and the accompanying emphasis on copyright issues, the takeaway may not be what you think.
They may end up seeing a millionaire artist trying to ban the voice of a small time voice actor.
Artists may actually get the short end of the stick in this in the general public perception.
Too often the choir of criticism heard in media outlets is confused for the actual public opinion. Trump was broadly ridiculed and dismissed and then won an election to the complete surprise of (I'd say) most people in the media business. I'm not gloating over Trump winning that election, I'm saying it is dangerous to assume you're correct because the people around you think so. It always sucks to be wrong.
This may be a similar thing.
And I think I may not be the only one to realize it. Many outlets stopped covering ScarJo's case and immediately moved on to talking about the larger picture and every other wrong found with OpenAI.
That is not illogical, but the public is not stupid. OpenAI allowed plenty time for the initial outrage to swell, I think purposefully.
This may actually still be a win for OpenAI.
They may actually end up reinstating the Sky voice (perhaps with a forced disclaimer it is not ScarJo and a disingenuous apology).
But regardless further coverage of this specific case from now on probably helps, not hurts, OpenAI.
Social ownership of these companies may be the only possible way forward. Fundamentally, such hugely powerful organisations should not be in the hands of a small few to do with as they please, so ownership should be equalised throughout society. All of us built these companies with our data, whether we knew it or not at the time, so there's an argument that we deserve a stake in them. Even disregarding that argument, whoever ends up wielding this technology will have immense, potentially at some point godlike, power. So it should be humanity as a whole (or at the very least, the democracies of the world) who collectively own and democratically govern such power. Just as political power should not be authoritarian, neither should economic or technological. How exactly this ownership model would pan out I'm not sure, would love anyone's thoughts on that.
" In the past, when people with power overstepped boundaries, people could enact physical resistance and usurp them," If there were a concerted effort to take down the power grid, would that not be physical resistance? I am not sure that assuming the current flow, no pun intended, continues apace is valid. Might it suggest a blind spot in the analysis?
My thoughts on your argument are fairly instinctual. I may feel quite differently after I've had more time to process. Initialy this feels as fear-mongering and baseless to me as people who are claiming AGI will end humanity.
Your argument seems to me to have the following premises:
1. All companies developing AI or AGI work together closely, have the same goals, and are not in competition. I.e. company A will not challenge the actions or authority of company B.
2. All employees from those companies are working towards the same goals or can be controlled into doing so. I.e. employee A of company A will not whistleblow.
I don't know about 1, but that seems dubious to me.
But 2 is definitely not true. Remember, we know about many of the problems in OpenAI because of the employees there who have stood up and said "hey! There's a problem here!". You have members of the organisation to thank for some of that information. There will be more who raise alarm bells, what we MUST do is listen to their specific concerns. Not fear monger. They are the people working directly on the real problems.
I do not understand any of these arguments here about closed vs open source.
Open source will continue. We already have open source and freely available LLMs, they are improving and we have projects such as gpt4all as attempts to get these running locally on consumer grade hardware.
Closed source will continue. Companies like Microsoft and Google aren't suddenly going to open source all of their products and/or AI models. And they aren't going to be forced to either, that wouldn't make economic sense.
The system will continue just as it has, there is no evidence to suggest that is changing. The algorithms, equations and techniques here are available to all of humanity. The data and compute are not, but there is an awful lot of freely available data. Compute is tied more closely to physical resources, so it is more of a struggle to get access to. I doubt there will be huge amounts of compute power made available to all of humanity, openly, any time soon.
Again, my caveat is that this are just my initial thoughts.
Oh and regarding the two premises I mentioned earlier, they were not explicitly stated by you but inferred from your argument. Specifically, the notion that there will be "no resistance" implies that there would be no competition between companies or whistleblowing from within companies. Both competition and whistleblowing can be seen as avenues of potential resistance. Hence, my argument focused on these aspects to highlight that there are indeed multiple axes of resistance historically and currently, which contradicts the idea of absolute control by any single entity.
Johnathan, thank you for your detailed response. I'd like to clarify a few things.
Your analogy to heart surgery is noted, but I believe the AI debate is a public discourse where varied perspectives are important. Unlike heart surgery, AI affects us all and we all have to contend with AI powered software. We do not all have to do heart surgery. This is a false equivalence. Criticism and discussion help refine understanding. I was not dismissing your arguments but rather providing my initial perspective in what I thought was an open forum.
You mentioned my lack of understanding of the closed vs. open-source debate. I merely meant that I did not understand why, further up in the comments, there was a debate on whether AI should be closed source or open source. It is both and will continue to be both. I should clarify that my point was that both models will continue to coexist, each serving different purposes. So it seemed, as evidenced with other software, unlikely there will be a closed monopoly on AI. If you have evidence to the contrary, by all means present it.
My reference to "instinctual" was about my own initial thoughts, not your argument. I appreciate that your stance is based on perception and historical actions of these companies. I apologise for not being clearer here and accidentally implying your arguments were instinctual. I also retract the comment that it seemed baseless, I spoke out of turn, I just disagree with your basis.
Regarding fear-mongering, my point was about the tone of the argument potentially instilling fear in others. Fear-mongering involves attempting to create fear, which doesn’t require you personally to be fearful (i.e., it doesn’t really matter if you fear those companies or not).
You stated, "THERE WILL BE NO RESISTANCE THAT CAN OPPOSE THEM" if we let those companies continue the way they are. I appreciate that this is not the tone of all your other comments; it just stood out to me in this train of thought. It is difficult to read comments about this new form of tyranny and not view it as fear-mongering. Thank you for clarifying that it wasn't intended that way.
However, I'd add that history shows that resistance can arise from within companies and between competitors. Whistleblowers, internal dissent, and competition between companies like Microsoft, Google, Meta, and others can serve as checks on each other's power as long as we oppose monopolies. Additionally, historically, governments and regulatory bodies can and do impose restrictions. While these companies have substantial power, the idea that there will be a point of no resistance to a private company is not supported by historical evidence. I accept this is far from a perfect system, free of manipulation, but the idea that AI will enable these companies to create a new form of tyranny that cannot be resisted is not supported by historical data any more than the idea that AI is an existential threat. To put it plainly, it is uncharted territory.
I am not saying that I trust these companies. I do not. I believe regulation will be necessary. Just that your argument (in this particular thread) seems extreme, and that tyranny through AI and private enterprise is a new concern. Afterall you said this will be "the birth of a new kind of Tyranny that was only envisioned in sci-fi novels" which didn't seem compatible with the idea of historical precedent.
I haven't noticed much in the way of ethical behaviour from Google; rather the reverse if anything. Their AI project may currently be less bad than OpenAI, but I expect that it will regress to the Google norm in time.
The hammer needs to hit them all.
Maybe not Anthropic.
Open source development is DEFINITELY NOT the way to safety. Its bad enough having to corral corporations without potentially handing extremely powerful technologies to rival governments, terrorists and others like accelerationists who explicitly seek the end of humanity.
Anthropic has at least produced good research, which can be used to prevent or lower the risk of AI takeover. In fact, in terms of capabilities versus safety, they've delivered safety more than anyone else.
https://time.com/6980210/anthropic-interpretability-ai-safety-research/
Shooting yourself in the face to spite your nose is no way to survive a future, especially with all of humanity on the line with us. We also don't open source nuclear weapons, and biological weapons for reason.
If you have low expectations you're hard to disappoint, so I get it - the world is a rough place. At the same time I think reverting to cynicism about the company as a whole is unfair to the AI team who are genuinely doing good things.
Anything can of course still happen in the long run but I believe good efforts deserve recognition and praise, otherwise you're contributing to a world where eventually nobody sees the point in doing any good.
I'm retired from working in digital technology, and have consequently seen far too much of how the sausage tends to be made.
In particular, I've seen a metric ton of hypocrisy - claiming one thing while doing another. This usually comes from the top down, and some of the youngest software engineers believe and are inspired by the idealistic hype, but others understand they are supposed to "say what I say, but do what I do," with the two wildly inconsistent.
Obviously I haven't worked for every single tech company, and in particular have not personally worked for Google. (But I know people who have, and it sounds like a normal big tech company to me.) There are plenty of decent people there, behaving decently except when instructed otherwise, and sometimes even working around their instructions. That's true almost everywhere. But it tends not to be enough to prevent overall bad effects, when incentives skew towards encouraging bad behaviour, particularly when the bad behaviour is at the expense of strangers (such as the general public, future generations, etc.)
I agree but like you said this is mostly the same everywhere.
I still think it is better to have a slogan that says "do no evil" versus "let the world burn".
It will impact the people you attract as you said and therefore your company, whether the board likes it or not.
If you're point is that it is sad that people will be disappointed, face disillusion, feel misled and may end up cynical I agree.
But I don't think changing the slogan to "let the world burn" is a constructive solution even though - once you end up cynical - it may feel more appropriate. But that is also because a part of cynicism is that it means you have given up and might want to see things burn out of spite.
The biggest challenge in life is to stay optimistic and hopeful and strive for the best. You will end up disappointed many times but it can still be the best strategy with the best outcomes overall.
This is also why I think having the same people in power too long is risky. Cynism easily leads to corruption because why should you continue to care of time and time others disappoint you?
This is not to discriminate against old people individually, just to say it's challenging to shed naivety without replacing it with cynicism. To keep trying and striving is exhausting and deserves respect.
I am glad that there are people like you still trying to improve human behaviour. Without those efforts, things would be even worse than they actually are.
Balance is required - history is littered with attempts to create perfection, that turned into particularly nasty oppressive regimes. It's also littered with martyrs, many (most?) to causes that never succeeded.
But it's also full of real improvements, both short term (something was good for a while) and long term (e.g. the very long term trend of reduction of violent death - see Pinker if this is unfamiliar).
Each person gets to judge for themselves in each particular case whether to tilt against those particular windmills/try to right those particular wrongs.
Those choices are partly a matter of personality, partly of their personal situation, and partly of experience. Age is probably also a factor, as you suggest. (I'm not sure whether my current cynicism is due to specific experiences, generational experiences - the 1960s were more optimistic than the 2020s - or personality.)
Yeah I wasn't trying to say you as a person are very cynical - even if you displayed some cynicism.
Sometimes we have to vent or are in a bad mood. Everyone is cynical about things sometimes. I know I've posted some rants here and there in my time. It depends also on what occupies you at any given time.
Maybe that's the good thing, that negativity and cynicism can also weather with time.
Annoying that you can't edit out spelling errors. I obviously meant your* instead of you're point.
FWIW, Substack has the ability to allow users to edit their own replies for some period of time after making them. Marcus has this enabled, you get to it via the horizontal row of three dots on the far right of the row with like, reply and share buttons.
Don't forget they also collaborated on enforcing the great firewall of China
No I was referring to Google.
The story with Google and China is complicated and goes back many years. They've flip-flopped a few times now about their approach to working (or not) with the CCP. But there was a time when they enforced the censorship in the 00's, and they were working on a new censored search product as of 2018 strictly for the Chinese market. Background here:
https://theconversation.com/googles-censored-chinese-search-engine-a-catalogue-of-ethical-violations-101046
All of this is probably a distraction from the fact that they now do *domestic* censorship as well (in more subtle ways, deranking, etc).
They are hanging themselves via there own words!! Impressive investigation Gary. I tip my hat. Much respect. This I hope decrease the haze that these companies seem to cause people. #beautiful
- This needs to be everywhere
Yes the public will be convinced this but some will think for themselves. What I call a 'distinct minority'. It is the mindset I focus on. I like your assessment.
Worrying about OpenAI 'winding up first to AGI' kind of suggests they potentially are on a road there. Do you really want to entertain that suggestion?
I don't think Gary has ever said otherwise, or suggested AGI wasn't possible? I'd guess he thinks the mental models of high-ups at OpenAI are wrong though. In the past, Sam Altman has acted in a way consistent with the hypothesis that he thought scaling up GPTs larger and larger would lead directly to AGI, and Gary has always (correctly) said it won't. But there are tons of smart people at OpenAI, and as long as the money keeps rolling in I expect them to eventually make some important technical breakthroughs toward AGI, which they say is their only mission after all.
Yep exactly, I had the same question: https://garymarcus.substack.com/p/scarjo-is-just-the-tip-of-the-iceberg/comment/57134702?r=32d6p4&utm_medium=ios
Mr Marcus, in many of your posts, you mention current efforts as a dead-end on the road to AGI (I agree with you there). But in some other posts like this one, you mention “who gets to AGI first” which seems as if the road actually leads there after all. Where do I misunderstand you please, could you help me clarify?
I think it’s meant rethorical: a company, any company for that matter, that has AGI as its mission should be held at the highest standards and be given maximum scrutiny.
From this perspective it indeed makes sense, thank you!
I was also a bit taken aback by that remark.
Everything is reduced to the cash nexus - as Marx helpfully pointed out. Thanks for fighting on Gary, And for keeping the rest of us in the loop.
Didn’t they release proof they didn’t use ScarJos voice? I’ve only seen the headlines haven’t had a chance to dig in!
they didn’t use it but they pretty clearly tried to impersonate her, and legal precedent in Midler vs Ford means that’s a no go.
All this fuss about OpenAI is a distraction. This is a quiet period before piracy storm.
Soon we’ll have AI chips in consumer devices and software that will provide user friendly ways for training LLMs for your own personal use.
And then it will be same as Napster days, but instead of MP3s we will download (unofficial) snapshots of Reddit, Quora, StackOverflow, etc. and train your personal AI.
And you will be able to take movie/tv show recordings and train your personal AI to be any actor or celebrity.
Remember old days of TomTom navigator and how you could download spoken directions by celebrities (imitations) despite them never recorded those?
Nobody is talking about this upcoming storm.
Good point. If I may be optimistic for a moment, this desire to be closer to the celebrity in question points to the powerful social draw that underlies how we respond to celebrity. That makes celebrity (and the artistry that creates it) intrinsically hugely valuable. AI is never going to replace that; "I'm a huge fan of Stockfish" just isn't a thing even if it can defeat Magnus Carlsen every game it plays. People with talent will continue to earn good-to-great livings even if they don't benefit every time someone clicks. As Oliver Wilde quipped: "there is only one thing worse than being talked about, and that is not being talked about".
I don't think we have to worry about OpenAI finding their AGI Holy Grail. They danger is in them successfully convincing others to go sticking AI in all sorts of places it doesn't belong. That's the business model for almost all of the consumer-products industry.
I think since that whole debacle back in Nov, Sam Altman is now in his "I dont trust anyone anymore" (Lex Fridman pod) phase. As if he think he was so naive before believing in ethical governance or something. He has a marvel baddie look about him now he's been burnt......
I think one needs to define what "benefiting humanity" means exactly because as it is it's too loose. As an example, for the Chinese government it might mean better algorithms to detect faces and identify people. For me, the best thing would be for it go back into the hole it crawled from.
Reading between the lines, it seems to mean "make Star Trek come true"
Well, we don't seem to be getting to Star Trek sadly but Matrix with Idiocracy.
I think ScarJo is not a mistake but a concerted effort to lure an Artist into challenging the fact that someone can have a similar voice.
If so it is a devious trap. They made it so it was very believable they did steal her voice. But they also primed us (and her!) to hear her own voice.
It is hard to say whether the strategy works long term but you have to remember that they're not after convincing the already vocal critics that you may be surrounded by.
To the general public that may not share what are essentially niche concerns and the accompanying emphasis on copyright issues, the takeaway may not be what you think.
They may end up seeing a millionaire artist trying to ban the voice of a small time voice actor.
Artists may actually get the short end of the stick in this in the general public perception.
Too often the choir of criticism heard in media outlets is confused for the actual public opinion. Trump was broadly ridiculed and dismissed and then won an election to the complete surprise of (I'd say) most people in the media business. I'm not gloating over Trump winning that election, I'm saying it is dangerous to assume you're correct because the people around you think so. It always sucks to be wrong.
This may be a similar thing.
And I think I may not be the only one to realize it. Many outlets stopped covering ScarJo's case and immediately moved on to talking about the larger picture and every other wrong found with OpenAI.
That is not illogical, but the public is not stupid. OpenAI allowed plenty time for the initial outrage to swell, I think purposefully.
This may actually still be a win for OpenAI.
They may actually end up reinstating the Sky voice (perhaps with a forced disclaimer it is not ScarJo and a disingenuous apology).
But regardless further coverage of this specific case from now on probably helps, not hurts, OpenAI.
Social ownership of these companies may be the only possible way forward. Fundamentally, such hugely powerful organisations should not be in the hands of a small few to do with as they please, so ownership should be equalised throughout society. All of us built these companies with our data, whether we knew it or not at the time, so there's an argument that we deserve a stake in them. Even disregarding that argument, whoever ends up wielding this technology will have immense, potentially at some point godlike, power. So it should be humanity as a whole (or at the very least, the democracies of the world) who collectively own and democratically govern such power. Just as political power should not be authoritarian, neither should economic or technological. How exactly this ownership model would pan out I'm not sure, would love anyone's thoughts on that.
Google keeps using The Onion in its search results https://open.substack.com/pub/readtpa/p/googles-ai-generated-search-results-9e1?utm_source=share&utm_medium=android&r=1uotn7
"You Either Die A Hero, Or You Live Long Enough To See Yourself Become The Villain"
Is there scope for the IRS to reassess this tax-free status and hit them with retrospective tax and penalties?
" In the past, when people with power overstepped boundaries, people could enact physical resistance and usurp them," If there were a concerted effort to take down the power grid, would that not be physical resistance? I am not sure that assuming the current flow, no pun intended, continues apace is valid. Might it suggest a blind spot in the analysis?
My thoughts on your argument are fairly instinctual. I may feel quite differently after I've had more time to process. Initialy this feels as fear-mongering and baseless to me as people who are claiming AGI will end humanity.
Your argument seems to me to have the following premises:
1. All companies developing AI or AGI work together closely, have the same goals, and are not in competition. I.e. company A will not challenge the actions or authority of company B.
2. All employees from those companies are working towards the same goals or can be controlled into doing so. I.e. employee A of company A will not whistleblow.
I don't know about 1, but that seems dubious to me.
But 2 is definitely not true. Remember, we know about many of the problems in OpenAI because of the employees there who have stood up and said "hey! There's a problem here!". You have members of the organisation to thank for some of that information. There will be more who raise alarm bells, what we MUST do is listen to their specific concerns. Not fear monger. They are the people working directly on the real problems.
I do not understand any of these arguments here about closed vs open source.
Open source will continue. We already have open source and freely available LLMs, they are improving and we have projects such as gpt4all as attempts to get these running locally on consumer grade hardware.
Closed source will continue. Companies like Microsoft and Google aren't suddenly going to open source all of their products and/or AI models. And they aren't going to be forced to either, that wouldn't make economic sense.
The system will continue just as it has, there is no evidence to suggest that is changing. The algorithms, equations and techniques here are available to all of humanity. The data and compute are not, but there is an awful lot of freely available data. Compute is tied more closely to physical resources, so it is more of a struggle to get access to. I doubt there will be huge amounts of compute power made available to all of humanity, openly, any time soon.
Again, my caveat is that this are just my initial thoughts.
Oh and regarding the two premises I mentioned earlier, they were not explicitly stated by you but inferred from your argument. Specifically, the notion that there will be "no resistance" implies that there would be no competition between companies or whistleblowing from within companies. Both competition and whistleblowing can be seen as avenues of potential resistance. Hence, my argument focused on these aspects to highlight that there are indeed multiple axes of resistance historically and currently, which contradicts the idea of absolute control by any single entity.
Johnathan, thank you for your detailed response. I'd like to clarify a few things.
Your analogy to heart surgery is noted, but I believe the AI debate is a public discourse where varied perspectives are important. Unlike heart surgery, AI affects us all and we all have to contend with AI powered software. We do not all have to do heart surgery. This is a false equivalence. Criticism and discussion help refine understanding. I was not dismissing your arguments but rather providing my initial perspective in what I thought was an open forum.
You mentioned my lack of understanding of the closed vs. open-source debate. I merely meant that I did not understand why, further up in the comments, there was a debate on whether AI should be closed source or open source. It is both and will continue to be both. I should clarify that my point was that both models will continue to coexist, each serving different purposes. So it seemed, as evidenced with other software, unlikely there will be a closed monopoly on AI. If you have evidence to the contrary, by all means present it.
My reference to "instinctual" was about my own initial thoughts, not your argument. I appreciate that your stance is based on perception and historical actions of these companies. I apologise for not being clearer here and accidentally implying your arguments were instinctual. I also retract the comment that it seemed baseless, I spoke out of turn, I just disagree with your basis.
Regarding fear-mongering, my point was about the tone of the argument potentially instilling fear in others. Fear-mongering involves attempting to create fear, which doesn’t require you personally to be fearful (i.e., it doesn’t really matter if you fear those companies or not).
You stated, "THERE WILL BE NO RESISTANCE THAT CAN OPPOSE THEM" if we let those companies continue the way they are. I appreciate that this is not the tone of all your other comments; it just stood out to me in this train of thought. It is difficult to read comments about this new form of tyranny and not view it as fear-mongering. Thank you for clarifying that it wasn't intended that way.
However, I'd add that history shows that resistance can arise from within companies and between competitors. Whistleblowers, internal dissent, and competition between companies like Microsoft, Google, Meta, and others can serve as checks on each other's power as long as we oppose monopolies. Additionally, historically, governments and regulatory bodies can and do impose restrictions. While these companies have substantial power, the idea that there will be a point of no resistance to a private company is not supported by historical evidence. I accept this is far from a perfect system, free of manipulation, but the idea that AI will enable these companies to create a new form of tyranny that cannot be resisted is not supported by historical data any more than the idea that AI is an existential threat. To put it plainly, it is uncharted territory.
I am not saying that I trust these companies. I do not. I believe regulation will be necessary. Just that your argument (in this particular thread) seems extreme, and that tyranny through AI and private enterprise is a new concern. Afterall you said this will be "the birth of a new kind of Tyranny that was only envisioned in sci-fi novels" which didn't seem compatible with the idea of historical precedent.