Oct 16, 2023·edited Oct 16, 2023Liked by Gary Marcus
Precisely - As I was reading MA's version, I was thinking about - so what IS the problem that we are addressing with AI? It's current development, investment paradigm, and application space is not overwhelmingly inclined towards the ESGs, or SDGs, or the essence of what we need for civilizational or human flourishing like nurturing and friendship. I too am a techno-optimist, but this kind of blind rant of the economic upsides is precisely what I writhe against. We need to WIDE and EXPANSIVE metrics that we ideate across and measure our progress against, not NARROW and singular ones such as economic market upsides. It's an extractive dynamic that has only one possible outcome - extinction. This moment calls for a wise re-imagining, not emotional tirades praising the shiny (profitable) worthiness of intelligence at scale.
Oct 17, 2023·edited Oct 17, 2023Liked by Gary Marcus
Haha, It's basically a foundational myth. It reads like Plato's Republic where he advocates lying about the founding of the state, so the philosopher kings can be seen in a better light. But in this case, we can swap the republic for surveillance capitalism. The philosopher kings are his list of names at the bottom. No time for sources!
It’s also ahistorical. I’ve worked in Internet policy for Internet orgs. I still, on an optimistic day, see the positives of the Internet. But it was started as a military research project, right? https://www.darpa.mil/about-us/timeline/arpanet
Yeah, invented at DARPA, by people that didn't want to or care to make money off it. Most people didn't think the internet would be that big of a thing either. In fact, I seriously doubt anyone had a close approximation of what it would become. Bill Gates, Paul Krugman, and lots of others thought it would just be a slight improvement over the fax machine in the early to mid 1990's.
Oct 16, 2023·edited Oct 16, 2023Liked by Gary Marcus
I'm not a Marxist, myself. Many of his ideas have simply not survived the test of time. Mixed market economy with a strong social safety net for the win, at least for the foreseeable future.
That said, it's interesting how Marc identifies communists as "the enemy" when Marx was one of the first techno-optimists. If anything, I would call that very techno-optimism one of Marx's weaknesses, but either way, automation and technological progress were key to his vision of post-capitalism. ("Capitalism was the best and worst thing to ever happen", in the memorably pithy phrasing of Marxist Fredric Jameson.) Some communists, those I can work with, still remember this tendency of their teacher. Others, like the de-growthers (who fail to realize their project would be a humanitarian disaster) follow a left-wing line of thought born as a reaction to and rejection of both canonical Marxism and liberalism.
For, like, 90% of the world, the delineations between these three squabbling factions with rather porous borders are academic at most. Don't worry if your eyes glazed over. But if you're going to be writing broad, grandiose manifestos, it helps to at least have at least some passing, 101-level familiarity with one of the most influential thinkers of the 19th Century... even if only to better oppose him and and his followers.
There's also a modern strain of communism called "Fully Automated Luxury Communism", which is a pro-tech for of communism. Communists barely exist in America, though. It's yet another screed with made up enemies, and no references of actual quotes or people that he seems to be at war with.
You're assuming Andreessen actually bothers to think about what he writes. He's just part of long line of rich boys with nothing better to do than blurt out whatever brain fart manifests itself. I doubt he's read a single academic book in his life.
These sorts of documents are interesting. One would have thought postmodernism would have killed the genre of the manifesto off, but it lives on. Who precisely is the “we” of these documents? I sense the author is hoping to imagine a we into existence, capture an audience or moment with his confident and unsubstantiated claims. The feel of this document is very naive and utopian and lands in a horribly disjointed way in light of the humanitarian crisis in the Middle East. What is AI going to do to solve conflicts rooted in thousands of years of human history? I digress... how about a different genre? Just a simple proposal containing concrete next steps on how to best use these tools. There is too much big picture work being done right now when we need more people leaning in the small projects of figuring out the here and now. At least that is how i feel about things today.
Also posthumously signing the names of artists and their philosophers who may or may not agree with all the tenets is just downright un-cool, to put it technically.
Brad DeLong's still around. Haven't read him in years, but if anyone's got an "in", it might be funny to see how he feels about being included in this fellow's "patron saints" list.
If I didn't know who the author was I would imagine this was a spoof of the communist manifesto (ah, the irony, as the enemy are branded as communists). This is not surprising, since both manifestos have the same political goals, and therefore follow a similar strategy and structure of manipulation to implant ideas into the unthinking heads of potential followers (aka useful idiots). Both manifestos start with setting the stage by presenting themselves as the victims - technology is persecuted in one, working people are persecuted in the other. Then they say who they are - techno-optimists in one, communists in the other; and proceed to define their believes and brand their enemies. They both use a factually correct central idea (that technology brings progress, in one, and that there is class struggle in society, in the other) that is then twisted to serve as the pivot of their ideology - that unbridled technological development and absolutely free market is the way forward, in one, and that communism and rule of the working class is the solution, in the other. Both manifestos are just so hilariously similar in structure, style and content, and both manifestos obviously seek a similar end, and we painfully well know from history what that end is.
I am not going to comment on all the inaccuracies and blatant propaganda as that would require an equally or even bigger volume of text as the manifesto itself. But the central point that technology is a universal source of good is obviously misguided - technology is just an enabler, whether it is used for good or bad depends on the people who operate it, and I have very high mistrust in the motives of the people that stand behind that manifesto.
Thanks! Having grown up in what was then a communist country, I suppose I might have an overdeveloped sense for and intolerance towards ideological propaganda.
Oct 17, 2023·edited Oct 17, 2023Liked by Gary Marcus
Man his essay really is something. I doubt many people here have enough knowledge of history and actual economics to see how many errors are in it. It's a worthwhile read just to understand how much of a fantasy bubble these silicon valley billionaires live in. I'll write up a tiny fraction of the errors below.
Firstly, I do not see any actual quotes from anyone that is anti-technology. All I see is straw-men (The unabomber recently passed away). However, he does cite "bad ideas":
Some evil ideas that he cites:
“existential risk”,
“sustainability”,
“Sustainable Development Goals”,
“social responsibility”,
“trust and safety”,
“risk management”,
“de-growth”
Firstly, none of these views are anti-technology. They are actually pro-tech, all of them want more investment in green energy technology. Free market capitalism never gave rise to solar energy. it was developed in a capitalist monopoly, ATAT's bell labs, and could not find a market outside of space ships for a very long time. That's a failure of the free market. Not many venture capitalists want to invest in something thats only viable after several decades of RandD.
Secondly, saying social responsibility, trust, safety, and risk management are the bad ideas kinda exposes alot about your moral compass. It sounds somewhat sociopathic, right?
It should be noted that de-growth is actually often about reducing material extraction, not economic growth per say, at least thats what Jason Hickel has said. His book "less is more" actually promotes investment into green energy, but also advocates reduced consumption and advertising with combined investment. But he has specifically said that de-growth isn't actually about reducing GDP growth, rather it's about reducing material footprint and extraction.
Oct 17, 2023·edited Oct 17, 2023Liked by Gary Marcus
So, he cites Bertrand Russell as an ally. I remember reading Russells work in high school. He was an advocate of libertarian socialism, in the strains of anarchism, syndicalism, and guild socialism. Russell actually thought it was kind of obvious that capitalism wouldn't last more than a few decades. Not a strange opinion to have after WW1 among the left. I believe Bertrand Russell wrote essays on how society had to reconcile politics and technology and that the decent society could not be achieved without both issues worked on.
One of the people he cites as an influence is a William Nordhaus. He got a nobel prize in economics. Do you know what he said in his lecture? That we should raise global temperatures to 3.5-4 degrees by 2150 (doing otherwise would reduce growth). His ideas are taught in econ101 now, and presented as progressive. Meanwhile, insurance companies estimate damages from climate change to be 10x+ greater than what Nordhaus did, in his 30 years of research.
The focus on automation as the reason for unemployment has been an attempt to deflect people away the reality that full employment could actually be achieved without much, if any rise in inflation. I first saw this under the Obama administration during the wake of the great recession, and this deflection has been repeated since then.
Three prominent economists have talked about this: Dean Baker, Robert Pollin, and the MMT crowd (Bill Mitchell, Randal Wray, Stephenie Kelton, ect...). More economists are coming around to the MMT world view and it looks like there may be a paradigm shift soon.
By the way, to achieve a planet of 50 billion people that is sustainable, well...economic efficiency gains average about 1.1% a year over the last century, economic growth is around 2% a year, and we are currently extracting enough raw material to be 4 times beyond the planetary boundaries by 2050. So, we would have to wait a long, long time, before technology was efficient enough for that to be possible, and efficiency gains would have to be more than growth in order to not destroy the whole eco system.
Oct 17, 2023·edited Oct 17, 2023Liked by Gary Marcus
Here some seriously wrong quotes from Marc's essay:
"Productivity growth, powered by technology, is the main driver of economic growth, wage growth, and the creation of new industries and new jobs, as people and capital are continuously freed to do more important, valuable things than in the past."
This has been false for several decades now. Under neoliberalism there has been a productivity/wage gap. Wages have not kept up at all with productitivity growth since the 1970's. This is entirely political, and not anything related to technology.
"We believe the ultimate moral defense of markets is that they divert people who otherwise would raise armies and start religions into peacefully productive pursuits."
So, the nordic model which pursued mixed economies over free markets is hardly turning into a theocracy. One could argue that Silicon Valley is a polythestic society however, when billionaires say that "we live in a simulation", or "AGI right around the corner", or "information theory explains all", or "free markets and tech solves all", or "imma try live long enough to download my mind onto a computer and acheive immortality. The last 1000+ attempts by rich people to achieve immortality in ancient times were all wrong but this time im right".
The free markets that poor countries were told to embrace had something called the "third world debt crisis". It was one of the times that zero growth was in fact achieved under capitalism. It was all in the name of comparative advantage as well, the idea that he cites as a big winner. No country has developed this way. They developed via the infant industries argument. This is known in developmental economics since before I was born.
Here's another example of zero growth. India's free market economy under British rule. Zero growth there, and it went from being one of the richest and most advanced economies to having a life expectency lower than cave man times.
"The more energy we have, the more people we can have, and the better everyone’s lives can be. We should raise everyone to the energy consumption level we have, then increase our energy 1,000x, then raise everyone else’s energy 1,000x as well."
Yeah not many organisms on planet earth would survive this, as it would likely raise global temperatures by about 10 degrees. This is the worst part of his essay. His lack of knowledge on anything ecologically related is mind-boggling.
I won't deal with the rest. His essay shows a total ignorance of economics and history, and in general has so many errors it would take a thesis to point them all out.
"We believe a Universal Basic Income would turn people into zoo animals to be farmed by the state. Man was not meant to be farmed; man was meant to be useful, to be productive, to be proud."
I have to stop at this point. I cannot bear to read this diatribe from someone who themselves supports surveillance capitalism in order to farm me (or rather my behavioral surplus, as Shoshona Zuboff calls it).
Re degrowth, I guess it matters a lot which degrowther you're thinking of. Helen Caldicott keeps talking about how Mozart composed by candlelight and we need to use way less energy (and above all, no nuclear) and I perceive this as the more common degrowth view.
Something also to be said about the important role of critics and (constructive) criticism, for any given technology - those who skirt it usually don't "win", in the long run - sure, they may temporarily boost their ego by pretending (and lying to themselves) that the problems raised by the critics don't exist. Well, good news is that these kinds of "hide your head in the sand" optimists eventually become the victim of their own blindness (though at the cost of others, unfortunately).
In a way, one can't really "cheat" their way out of our core problems, as sooner or later those problems would have to be addressed, and the longer we wait to address them the harder it gets, and the more sudden and dramatic the downfall of any given technology (and technologists) becomes.
So it's in the best interest of any given leader or company who wants to have long-term success, is to listen to this early feedback. And if they don't, well the joke is on them, as there's no harsher critic than the time itself.
I for one love the fact that the Marc Andreessen's of the world are finally saying the quiet part out loud...Or starting to at least. I will leave the body of his sophomoric nonsense for everyone else to pick over, but I find it is absolutely priceless that Andreessen Horowitz's corporate homepage now features a manifesto that approvingly quotes Italian Fascists. Not, mind you, recent vintage Italian Fascists like Giorgia Meloni but OG Italian Fascists like, say, Filippo Marinetti!
"Pride goeth before destruction and a haughty spirit before a fall." Let's hope that this is the beginning of the end for Andreessen's grift...
While I too appreciate that they're no longer hiding it (and how many people were involved in workshopping this, to ensure that fascists were included and the rhetoric of the Enemy who must die was said, but also not quite said), it's important to note that this is the next phase of the classic fascist communications / propaganda playbook. Make sure everyone sees and understands that the quiet part can be said out loud. Acquaint people with the loud part. Convert the words to violent actions.
Andreessen is mad. Pathetically so. His vision and definition of "growth" is on one hand terribly childish and reductive, on the other is the same cancer cells would give. I have seen that manifesto only now, but it is just more of the crap he posted some months ago and I, for pure coincidence just described here as "sociopathic Solaria, all the way down" https://mfioretti.substack.com/p/taking-ai-too-seriously-is-not-intelligent
My favorite part of the Manifesto is that one of the allies is a fictional character because (apparently) he didn’t dare name the author directly as she is too controversial...
This is an important rebuttal. My impression is that Marc has taken on a populist sort of role as of late. He is not interested in writing about deeper topics or conducting full examinations, but maximizing potential for as many people as possible. This involves leaving out important details and watering down his rhetoric. I believe that this is a strategic move with a wider goal in mind. He has become like Joe Rogan, Lex Fridman, etc. It’s not a bad move for someone in his position of influence, but it is hard to take him seriously without understanding his intentions.
Hear hear. Couldn't believe the libertarian claptrap when I read it. I imagine Punch cartoon from the 19th century where a wealthy industrialist says to a poverty stricken worker "I'm a technology optimist, are you?" The future is always a long way off.
Oh boy. There is so much naive belief in that piece.
Take "We believe markets are an inherently individualistic way to achieve superior collective outcomes".
From the 1980's on, a vision 'unfettered capitalism' has driven the world economy. The promise was that 'freeing almost everything' would in the end 'benefit almost *all*'. After almost a half century of this, can we finally conclude that this 'all' has turned out to be simply not true? In the US, for instance, where this movement has been strongest, productivity has doubled, but real wages have stagnated. The rich have become richer based on those profits, the rest has stagnated because they depend on wage-related income. The top 10% of US families now own 76% of wealth. The bottom 50% own just 1%.
Regardless of this being a bad or a good thing (that is an ethical choice: is it actually important to have a fair society?), can we finally simply conclude that the theory (that *everyone* profits from unfettered capitalism) at least has definitely been proven dead wrong? How much more data do we need to accept this?
People get sold 'lower taxes' and 'more freedom', but the weaker the collective side of the economy (including government), the more unfettered predatory practices become, and the fewer people get those benefits in a *real* sense. If we really want more people to have real — and not just the marketing of — freedom (i.e. '*all* benefit') what needs to be done? And — given globalisation — can something be done at all?
Andreesen quotes modern capitalism founder Adam Smith. I find it ironical that if you read Adam Smith's "The Wealth of Nations" he actually warns for the fact that entrepreneurs are not to be trusted with the advise they give on what is best for *all* (i.e. society) as they are in the end singularly focused on what is good for *themselves*.
"We believe the ultimate moral defense of markets is that they divert people who otherwise would raise armies and start religions into peacefully productive pursuits." First, entrepreneurs do everything to make a profit. Period. Including lying to sell more arms (Eisenhower's Military-Industrial Complex). Or lie to sell more tobacco. Or lie to sell more oil. Or lie to sell hyper-addictive opioids. They even may start religions (i.e. Scientology) to make a profit. In fact, the key lie that unfettered capitalists tell themselves and are convinced of is this lie that unfettered capitalism is by definition good for *all*.
"Our enemy is speech control and thought control – the increasing use, in plain sight, of George Orwell’s “1984” as an instruction manual." *Cough*. Surveillance capitalism. *Cough*. Google dropping "don't be evil" as it is bad for profit. Letting evil types manipulate elections for profit.
The free market is absolutely a good thing for innovation. It is important, nobody believes we can plan everything. But ignoring the (moral and collective) downsides or the conditions for it is hyper-naive.
And Kelly? Kurzweil? Really?
And nuclear fission and fusion to give us all 1000x as much energy to use? Fine (but realistic?). And suppose that works after several decades, does that mean you're going to forbid fossil fuels? Or is that against the maximally free entrepreneurial dogma?
Techno-optimism is not necessarily bad. I like technology. I like innovation. How can we be against improvement? But religious 'free marketism' is *very* bad. A free market is an essential part of what I would think is the ideal society. But one thing it is not in any way is 'moral'. That has to come from somewhere else.
I am reminded of Bonhoeffer's analysis that stupidity is much more dangerous than malice.
"Against stupidity we are defenseless. Neither protests nor the use of force accomplish anything here; reasons fall on deaf ears; facts that contradict one’s prejudgment simply need not be believed- in such moments the stupid person even becomes critical – and when facts are irrefutable they are just pushed aside as inconsequential, as incidental. In all this the stupid person, in contrast to the malicious one, is utterly self-satisfied and, being easily irritated, becomes dangerous by going on the attack." It is not a lack of intellect he is talking about: "There are human beings who are of remarkably agile intellect yet stupid, and others who are intellectually quite dull yet anything but stupid."
Feel free to unsubscribe. LeCun snipes at me all the time, undermining your alleged recitals of facts, and my predictions have been both public and correct.
I like what Gary has to say and also how he says it, even when I disagree with individual points. The problem is the reverse, as AI proponents:
- Claim that the Singularity is here every time a classifier correctly identifies a cat picture
- Never acknowledging significant failure in the AI systems nor that the progress has been minor and incremental
- Going as far as propagating silly conspiracies that self-driving cars are ready and capable and only shadowy government figures are holding their adoption back
- Constant petty sniping at anyone daring to point out that the Emperor is naked
I don't think this is entirely fair. I think Gary rightly points out the unrealistic assessments of those that are convinced that what GIA does is much more than what it actually fundamentally is capable of doing. And he does that by providing counterexamples. Which is — Popper and all — the right thing to do, even if that is not enough to shake the existing convictions/paradigm. But the counterexamples do irk people because they are opposed to strongly held convictions.
LLMs have improved a lot since transformers opened up the training of much more massive models around 2016-2017, that is true. I think Gary acknowledges that as well. But understanding how these systems work leads me to the conclusion that it is unlikely we will solve the more fundamental underlying issues with more training or ever larger models. For the latter: take the data from OpenAI's "Language Models are Few-Shot Learners" paper, and take the zero-shot (which is the baseline for these models) for e.g. Lambada in Figure 3.2. Did you notice the x-axis is logarithmic? Do you know how the picture of that (and other) tests look when you remove that visual sleight of hand? And what that means for the chances that growing the models even more is really going to help (apart from that there is no business case to do so)? Yes there *has been* progress. But that is no guarantee there *will be* more progress. And if you look how it works, I am pretty confident that chances are slim there will be substantial progress through increasing model sizes.
I want a lot more of this. Sensible, careful, constructive work.
Less about "LeCun is an LLM", "self-driving cars cars bite the dust", "I was right but they laughed at me", "I make this prediction [where prediction is a triviality]".
In short, separating the persona from the chronicle. The persona stays in the way.
Gary, I would like to know how you can be an optimist about the prospect of AI actually working. Suppose everyone starts working on neuro-symbolic AI, or G-flow nets, or whatever, and we eventually succeed in having real human level artificial intelligence. Do you really expect that, with the price of cognitive work, the last thing we have available to sell, going down the drain, the average person will be able sustain themselves? How? Do you really think our democracies are strong enough to maintain a situation in which the great majority of people are maintained by the system, "for free"?
"We had a problem of isolation, so we invented the Internet" is so awful it feels like it must be self-parody.
Precisely - As I was reading MA's version, I was thinking about - so what IS the problem that we are addressing with AI? It's current development, investment paradigm, and application space is not overwhelmingly inclined towards the ESGs, or SDGs, or the essence of what we need for civilizational or human flourishing like nurturing and friendship. I too am a techno-optimist, but this kind of blind rant of the economic upsides is precisely what I writhe against. We need to WIDE and EXPANSIVE metrics that we ideate across and measure our progress against, not NARROW and singular ones such as economic market upsides. It's an extractive dynamic that has only one possible outcome - extinction. This moment calls for a wise re-imagining, not emotional tirades praising the shiny (profitable) worthiness of intelligence at scale.
Haha, It's basically a foundational myth. It reads like Plato's Republic where he advocates lying about the founding of the state, so the philosopher kings can be seen in a better light. But in this case, we can swap the republic for surveillance capitalism. The philosopher kings are his list of names at the bottom. No time for sources!
It’s also ahistorical. I’ve worked in Internet policy for Internet orgs. I still, on an optimistic day, see the positives of the Internet. But it was started as a military research project, right? https://www.darpa.mil/about-us/timeline/arpanet
Yeah, invented at DARPA, by people that didn't want to or care to make money off it. Most people didn't think the internet would be that big of a thing either. In fact, I seriously doubt anyone had a close approximation of what it would become. Bill Gates, Paul Krugman, and lots of others thought it would just be a slight improvement over the fax machine in the early to mid 1990's.
He's writing for people in their early 20s. Understand the demographic.
I'd dare suggest Andreessen writes also, if not primarily, for investors so dumb to still trust him with their money.
Andreessen trusts Adam Neumann with his money for yet another real estate boondoggle - essentially a WeLive reboot, so there’s that.
Builders, Marco!
I'm not a Marxist, myself. Many of his ideas have simply not survived the test of time. Mixed market economy with a strong social safety net for the win, at least for the foreseeable future.
That said, it's interesting how Marc identifies communists as "the enemy" when Marx was one of the first techno-optimists. If anything, I would call that very techno-optimism one of Marx's weaknesses, but either way, automation and technological progress were key to his vision of post-capitalism. ("Capitalism was the best and worst thing to ever happen", in the memorably pithy phrasing of Marxist Fredric Jameson.) Some communists, those I can work with, still remember this tendency of their teacher. Others, like the de-growthers (who fail to realize their project would be a humanitarian disaster) follow a left-wing line of thought born as a reaction to and rejection of both canonical Marxism and liberalism.
For, like, 90% of the world, the delineations between these three squabbling factions with rather porous borders are academic at most. Don't worry if your eyes glazed over. But if you're going to be writing broad, grandiose manifestos, it helps to at least have at least some passing, 101-level familiarity with one of the most influential thinkers of the 19th Century... even if only to better oppose him and and his followers.
There's also a modern strain of communism called "Fully Automated Luxury Communism", which is a pro-tech for of communism. Communists barely exist in America, though. It's yet another screed with made up enemies, and no references of actual quotes or people that he seems to be at war with.
You're assuming Andreessen actually bothers to think about what he writes. He's just part of long line of rich boys with nothing better to do than blurt out whatever brain fart manifests itself. I doubt he's read a single academic book in his life.
These sorts of documents are interesting. One would have thought postmodernism would have killed the genre of the manifesto off, but it lives on. Who precisely is the “we” of these documents? I sense the author is hoping to imagine a we into existence, capture an audience or moment with his confident and unsubstantiated claims. The feel of this document is very naive and utopian and lands in a horribly disjointed way in light of the humanitarian crisis in the Middle East. What is AI going to do to solve conflicts rooted in thousands of years of human history? I digress... how about a different genre? Just a simple proposal containing concrete next steps on how to best use these tools. There is too much big picture work being done right now when we need more people leaning in the small projects of figuring out the here and now. At least that is how i feel about things today.
Also posthumously signing the names of artists and their philosophers who may or may not agree with all the tenets is just downright un-cool, to put it technically.
The manifesto is 'something', full of name-dropping of dead people that would never agree with him.
But that's Andreesen: full of quotes, full of himself, full of shit.
Brad DeLong's still around. Haven't read him in years, but if anyone's got an "in", it might be funny to see how he feels about being included in this fellow's "patron saints" list.
Hah. As I suspected, DeLong himself expresses concern (though nothing in-depth at this time): https://substack.com/@delongonsubstack/note/c-42084469
If I didn't know who the author was I would imagine this was a spoof of the communist manifesto (ah, the irony, as the enemy are branded as communists). This is not surprising, since both manifestos have the same political goals, and therefore follow a similar strategy and structure of manipulation to implant ideas into the unthinking heads of potential followers (aka useful idiots). Both manifestos start with setting the stage by presenting themselves as the victims - technology is persecuted in one, working people are persecuted in the other. Then they say who they are - techno-optimists in one, communists in the other; and proceed to define their believes and brand their enemies. They both use a factually correct central idea (that technology brings progress, in one, and that there is class struggle in society, in the other) that is then twisted to serve as the pivot of their ideology - that unbridled technological development and absolutely free market is the way forward, in one, and that communism and rule of the working class is the solution, in the other. Both manifestos are just so hilariously similar in structure, style and content, and both manifestos obviously seek a similar end, and we painfully well know from history what that end is.
I am not going to comment on all the inaccuracies and blatant propaganda as that would require an equally or even bigger volume of text as the manifesto itself. But the central point that technology is a universal source of good is obviously misguided - technology is just an enabler, whether it is used for good or bad depends on the people who operate it, and I have very high mistrust in the motives of the people that stand behind that manifesto.
Damn I wish I had thought to say that! Smart analysis.
Thanks! Having grown up in what was then a communist country, I suppose I might have an overdeveloped sense for and intolerance towards ideological propaganda.
Man his essay really is something. I doubt many people here have enough knowledge of history and actual economics to see how many errors are in it. It's a worthwhile read just to understand how much of a fantasy bubble these silicon valley billionaires live in. I'll write up a tiny fraction of the errors below.
Firstly, I do not see any actual quotes from anyone that is anti-technology. All I see is straw-men (The unabomber recently passed away). However, he does cite "bad ideas":
Some evil ideas that he cites:
“existential risk”,
“sustainability”,
“Sustainable Development Goals”,
“social responsibility”,
“trust and safety”,
“risk management”,
“de-growth”
Firstly, none of these views are anti-technology. They are actually pro-tech, all of them want more investment in green energy technology. Free market capitalism never gave rise to solar energy. it was developed in a capitalist monopoly, ATAT's bell labs, and could not find a market outside of space ships for a very long time. That's a failure of the free market. Not many venture capitalists want to invest in something thats only viable after several decades of RandD.
Secondly, saying social responsibility, trust, safety, and risk management are the bad ideas kinda exposes alot about your moral compass. It sounds somewhat sociopathic, right?
It should be noted that de-growth is actually often about reducing material extraction, not economic growth per say, at least thats what Jason Hickel has said. His book "less is more" actually promotes investment into green energy, but also advocates reduced consumption and advertising with combined investment. But he has specifically said that de-growth isn't actually about reducing GDP growth, rather it's about reducing material footprint and extraction.
So, he cites Bertrand Russell as an ally. I remember reading Russells work in high school. He was an advocate of libertarian socialism, in the strains of anarchism, syndicalism, and guild socialism. Russell actually thought it was kind of obvious that capitalism wouldn't last more than a few decades. Not a strange opinion to have after WW1 among the left. I believe Bertrand Russell wrote essays on how society had to reconcile politics and technology and that the decent society could not be achieved without both issues worked on.
One of the people he cites as an influence is a William Nordhaus. He got a nobel prize in economics. Do you know what he said in his lecture? That we should raise global temperatures to 3.5-4 degrees by 2150 (doing otherwise would reduce growth). His ideas are taught in econ101 now, and presented as progressive. Meanwhile, insurance companies estimate damages from climate change to be 10x+ greater than what Nordhaus did, in his 30 years of research.
The focus on automation as the reason for unemployment has been an attempt to deflect people away the reality that full employment could actually be achieved without much, if any rise in inflation. I first saw this under the Obama administration during the wake of the great recession, and this deflection has been repeated since then.
Three prominent economists have talked about this: Dean Baker, Robert Pollin, and the MMT crowd (Bill Mitchell, Randal Wray, Stephenie Kelton, ect...). More economists are coming around to the MMT world view and it looks like there may be a paradigm shift soon.
By the way, to achieve a planet of 50 billion people that is sustainable, well...economic efficiency gains average about 1.1% a year over the last century, economic growth is around 2% a year, and we are currently extracting enough raw material to be 4 times beyond the planetary boundaries by 2050. So, we would have to wait a long, long time, before technology was efficient enough for that to be possible, and efficiency gains would have to be more than growth in order to not destroy the whole eco system.
Here some seriously wrong quotes from Marc's essay:
"Productivity growth, powered by technology, is the main driver of economic growth, wage growth, and the creation of new industries and new jobs, as people and capital are continuously freed to do more important, valuable things than in the past."
This has been false for several decades now. Under neoliberalism there has been a productivity/wage gap. Wages have not kept up at all with productitivity growth since the 1970's. This is entirely political, and not anything related to technology.
"We believe the ultimate moral defense of markets is that they divert people who otherwise would raise armies and start religions into peacefully productive pursuits."
So, the nordic model which pursued mixed economies over free markets is hardly turning into a theocracy. One could argue that Silicon Valley is a polythestic society however, when billionaires say that "we live in a simulation", or "AGI right around the corner", or "information theory explains all", or "free markets and tech solves all", or "imma try live long enough to download my mind onto a computer and acheive immortality. The last 1000+ attempts by rich people to achieve immortality in ancient times were all wrong but this time im right".
The free markets that poor countries were told to embrace had something called the "third world debt crisis". It was one of the times that zero growth was in fact achieved under capitalism. It was all in the name of comparative advantage as well, the idea that he cites as a big winner. No country has developed this way. They developed via the infant industries argument. This is known in developmental economics since before I was born.
Here's another example of zero growth. India's free market economy under British rule. Zero growth there, and it went from being one of the richest and most advanced economies to having a life expectency lower than cave man times.
"The more energy we have, the more people we can have, and the better everyone’s lives can be. We should raise everyone to the energy consumption level we have, then increase our energy 1,000x, then raise everyone else’s energy 1,000x as well."
Yeah not many organisms on planet earth would survive this, as it would likely raise global temperatures by about 10 degrees. This is the worst part of his essay. His lack of knowledge on anything ecologically related is mind-boggling.
I won't deal with the rest. His essay shows a total ignorance of economics and history, and in general has so many errors it would take a thesis to point them all out.
"We believe a Universal Basic Income would turn people into zoo animals to be farmed by the state. Man was not meant to be farmed; man was meant to be useful, to be productive, to be proud."
I have to stop at this point. I cannot bear to read this diatribe from someone who themselves supports surveillance capitalism in order to farm me (or rather my behavioral surplus, as Shoshona Zuboff calls it).
Re degrowth, I guess it matters a lot which degrowther you're thinking of. Helen Caldicott keeps talking about how Mozart composed by candlelight and we need to use way less energy (and above all, no nuclear) and I perceive this as the more common degrowth view.
True. Admittedly I havn't read all of the de-growther points of view.
Something also to be said about the important role of critics and (constructive) criticism, for any given technology - those who skirt it usually don't "win", in the long run - sure, they may temporarily boost their ego by pretending (and lying to themselves) that the problems raised by the critics don't exist. Well, good news is that these kinds of "hide your head in the sand" optimists eventually become the victim of their own blindness (though at the cost of others, unfortunately).
In a way, one can't really "cheat" their way out of our core problems, as sooner or later those problems would have to be addressed, and the longer we wait to address them the harder it gets, and the more sudden and dramatic the downfall of any given technology (and technologists) becomes.
So it's in the best interest of any given leader or company who wants to have long-term success, is to listen to this early feedback. And if they don't, well the joke is on them, as there's no harsher critic than the time itself.
I for one love the fact that the Marc Andreessen's of the world are finally saying the quiet part out loud...Or starting to at least. I will leave the body of his sophomoric nonsense for everyone else to pick over, but I find it is absolutely priceless that Andreessen Horowitz's corporate homepage now features a manifesto that approvingly quotes Italian Fascists. Not, mind you, recent vintage Italian Fascists like Giorgia Meloni but OG Italian Fascists like, say, Filippo Marinetti!
"Pride goeth before destruction and a haughty spirit before a fall." Let's hope that this is the beginning of the end for Andreessen's grift...
Great note, Gary. Thank you.
While I too appreciate that they're no longer hiding it (and how many people were involved in workshopping this, to ensure that fascists were included and the rhetoric of the Enemy who must die was said, but also not quite said), it's important to note that this is the next phase of the classic fascist communications / propaganda playbook. Make sure everyone sees and understands that the quiet part can be said out loud. Acquaint people with the loud part. Convert the words to violent actions.
That is an excellent and important point.
Andreessen is mad. Pathetically so. His vision and definition of "growth" is on one hand terribly childish and reductive, on the other is the same cancer cells would give. I have seen that manifesto only now, but it is just more of the crap he posted some months ago and I, for pure coincidence just described here as "sociopathic Solaria, all the way down" https://mfioretti.substack.com/p/taking-ai-too-seriously-is-not-intelligent
My favorite part of the Manifesto is that one of the allies is a fictional character because (apparently) he didn’t dare name the author directly as she is too controversial...
This is an important rebuttal. My impression is that Marc has taken on a populist sort of role as of late. He is not interested in writing about deeper topics or conducting full examinations, but maximizing potential for as many people as possible. This involves leaving out important details and watering down his rhetoric. I believe that this is a strategic move with a wider goal in mind. He has become like Joe Rogan, Lex Fridman, etc. It’s not a bad move for someone in his position of influence, but it is hard to take him seriously without understanding his intentions.
Hear hear. Couldn't believe the libertarian claptrap when I read it. I imagine Punch cartoon from the 19th century where a wealthy industrialist says to a poverty stricken worker "I'm a technology optimist, are you?" The future is always a long way off.
Oh boy. There is so much naive belief in that piece.
Take "We believe markets are an inherently individualistic way to achieve superior collective outcomes".
From the 1980's on, a vision 'unfettered capitalism' has driven the world economy. The promise was that 'freeing almost everything' would in the end 'benefit almost *all*'. After almost a half century of this, can we finally conclude that this 'all' has turned out to be simply not true? In the US, for instance, where this movement has been strongest, productivity has doubled, but real wages have stagnated. The rich have become richer based on those profits, the rest has stagnated because they depend on wage-related income. The top 10% of US families now own 76% of wealth. The bottom 50% own just 1%.
Regardless of this being a bad or a good thing (that is an ethical choice: is it actually important to have a fair society?), can we finally simply conclude that the theory (that *everyone* profits from unfettered capitalism) at least has definitely been proven dead wrong? How much more data do we need to accept this?
People get sold 'lower taxes' and 'more freedom', but the weaker the collective side of the economy (including government), the more unfettered predatory practices become, and the fewer people get those benefits in a *real* sense. If we really want more people to have real — and not just the marketing of — freedom (i.e. '*all* benefit') what needs to be done? And — given globalisation — can something be done at all?
Here are some facts about the last half century in the US: https://www.epi.org/publication/decades-of-rising-economic-inequality-in-the-u-s-testimony-before-the-u-s-house-of-representatives-ways-and-means-committee/
Andreesen quotes modern capitalism founder Adam Smith. I find it ironical that if you read Adam Smith's "The Wealth of Nations" he actually warns for the fact that entrepreneurs are not to be trusted with the advise they give on what is best for *all* (i.e. society) as they are in the end singularly focused on what is good for *themselves*.
"We believe the ultimate moral defense of markets is that they divert people who otherwise would raise armies and start religions into peacefully productive pursuits." First, entrepreneurs do everything to make a profit. Period. Including lying to sell more arms (Eisenhower's Military-Industrial Complex). Or lie to sell more tobacco. Or lie to sell more oil. Or lie to sell hyper-addictive opioids. They even may start religions (i.e. Scientology) to make a profit. In fact, the key lie that unfettered capitalists tell themselves and are convinced of is this lie that unfettered capitalism is by definition good for *all*.
"Our enemy is speech control and thought control – the increasing use, in plain sight, of George Orwell’s “1984” as an instruction manual." *Cough*. Surveillance capitalism. *Cough*. Google dropping "don't be evil" as it is bad for profit. Letting evil types manipulate elections for profit.
The free market is absolutely a good thing for innovation. It is important, nobody believes we can plan everything. But ignoring the (moral and collective) downsides or the conditions for it is hyper-naive.
And Kelly? Kurzweil? Really?
And nuclear fission and fusion to give us all 1000x as much energy to use? Fine (but realistic?). And suppose that works after several decades, does that mean you're going to forbid fossil fuels? Or is that against the maximally free entrepreneurial dogma?
Techno-optimism is not necessarily bad. I like technology. I like innovation. How can we be against improvement? But religious 'free marketism' is *very* bad. A free market is an essential part of what I would think is the ideal society. But one thing it is not in any way is 'moral'. That has to come from somewhere else.
I am reminded of Bonhoeffer's analysis that stupidity is much more dangerous than malice.
"Against stupidity we are defenseless. Neither protests nor the use of force accomplish anything here; reasons fall on deaf ears; facts that contradict one’s prejudgment simply need not be believed- in such moments the stupid person even becomes critical – and when facts are irrefutable they are just pushed aside as inconsequential, as incidental. In all this the stupid person, in contrast to the malicious one, is utterly self-satisfied and, being easily irritated, becomes dangerous by going on the attack." It is not a lack of intellect he is talking about: "There are human beings who are of remarkably agile intellect yet stupid, and others who are intellectually quite dull yet anything but stupid."
https://www.linkedin.com/pulse/stupidity-versus-malice-gerben-wierda/
I agree that one needs a more nuanced treatment than Andreessen's, and by and large, your outlook is a good and sensible position.
What irks me however, about your approach, is:
- Claiming the sky is falling each time a fake picture is released
- Picking at every single LLM and art gen failure, but not acknowledging that these are progress, and things improved a lot since 5 years ago
- Going as far as cheering for failure of self-driving cars (bites the dust, eh)
- Making obvious "prophesies" and then patting yourself on the back as in "I was right, and I was builied"
- Your disagreements with LeCun can devolve to a petty feud, with you doing all the sniping.
Feel free to unsubscribe. LeCun snipes at me all the time, undermining your alleged recitals of facts, and my predictions have been both public and correct.
I like what Gary has to say and also how he says it, even when I disagree with individual points. The problem is the reverse, as AI proponents:
- Claim that the Singularity is here every time a classifier correctly identifies a cat picture
- Never acknowledging significant failure in the AI systems nor that the progress has been minor and incremental
- Going as far as propagating silly conspiracies that self-driving cars are ready and capable and only shadowy government figures are holding their adoption back
- Constant petty sniping at anyone daring to point out that the Emperor is naked
I actually like what you write. When you focus on issues.
We need sound and sensible analysis of how AI is progressing, and what the issues are. You are good at that.
But we need less posturing, fewer self-references, and such. It is not about you.
I don't think this is entirely fair. I think Gary rightly points out the unrealistic assessments of those that are convinced that what GIA does is much more than what it actually fundamentally is capable of doing. And he does that by providing counterexamples. Which is — Popper and all — the right thing to do, even if that is not enough to shake the existing convictions/paradigm. But the counterexamples do irk people because they are opposed to strongly held convictions.
LLMs have improved a lot since transformers opened up the training of much more massive models around 2016-2017, that is true. I think Gary acknowledges that as well. But understanding how these systems work leads me to the conclusion that it is unlikely we will solve the more fundamental underlying issues with more training or ever larger models. For the latter: take the data from OpenAI's "Language Models are Few-Shot Learners" paper, and take the zero-shot (which is the baseline for these models) for e.g. Lambada in Figure 3.2. Did you notice the x-axis is logarithmic? Do you know how the picture of that (and other) tests look when you remove that visual sleight of hand? And what that means for the chances that growing the models even more is really going to help (apart from that there is no business case to do so)? Yes there *has been* progress. But that is no guarantee there *will be* more progress. And if you look how it works, I am pretty confident that chances are slim there will be substantial progress through increasing model sizes.
I want a lot more of this. Sensible, careful, constructive work.
Less about "LeCun is an LLM", "self-driving cars cars bite the dust", "I was right but they laughed at me", "I make this prediction [where prediction is a triviality]".
In short, separating the persona from the chronicle. The persona stays in the way.
I am saying this politely: I want less of you on me
Andreesen is moving the Overton window because he suffers not at all for doing so.
At the same time he’s drawing all your fire. I’m sure additional enemy lists are being compiled. So be it.
Gary, I would like to know how you can be an optimist about the prospect of AI actually working. Suppose everyone starts working on neuro-symbolic AI, or G-flow nets, or whatever, and we eventually succeed in having real human level artificial intelligence. Do you really expect that, with the price of cognitive work, the last thing we have available to sell, going down the drain, the average person will be able sustain themselves? How? Do you really think our democracies are strong enough to maintain a situation in which the great majority of people are maintained by the system, "for free"?
We cannot all be plumbers.