37 Comments

It's astonishing that so many people who work in AI don't actually seem to know anything about AI.

Expand full comment

Well, I have published an article with a rather long title that comments on that fact, or rather, a closely related matter, human language and cognition:

Aye Aye, Cap’n! Investing in AI is like buying shares in a whaling voyage captained by a man who knows all about ships and little about whales

https://3quarksdaily.com/3quarksdaily/2023/12/aye-aye-capn-investing-in-ai-is-like-buying-shares-in-a-whaling-voyage-captained-by-a-man-who-knows-all-about-ships-and-little-about-whales.html

Expand full comment

Investment is another animal. Uber is the poster child for investing in bullcrap. As long as you can keep the hype going while being technically accurate in your official SEC disclosures, you're golden.

Just tell them in the appropriate caveats section that you have no business plan for getting into the black and you do not project one. Voila! It's a mania, not a scam.

Expand full comment
6dEdited

Wonderful post, thanks for sharing that. The standard Hintonian response to your Shakespeare model would be "linguists spent decades making these things, and they didn't work as a tool for getting computers to talk like people, but deep learning models did work, and are therefore a more plausible mechanism for how humans talk". I've heard variants of this so many times in the last couple of years. The "therefore" doesn't actually connect the premises to the conclusion, but I guess some people have a really strong gut feeling that it ought to.

You say that AI experts see a dazzling city full of buildings with no doors. I like the analogy. The more responsible experts are happy to admit that they have no idea what's inside those things that look like buildings. The Hintons, though, think they're justified in assuming that what's inside looks like the device they used to create the facades. They might as well be intuiting that inside of a grand piano lies a digital amplifier and a MIDI controller and a bunch of electronic frequency filters, cos that's what they found in their Casio.

I wasn't familiar with the Pinker vs. Aaronson debate; I'm excited to check it out!

Expand full comment

Thanks. I did that work on the Shakespeare sonnet back in the mid-1970s, which was when symbolic AI was flourishing. Note, however, that I wasn't doing AI. I was doing computational linguistics (CL). Two different disciplines, with different (institutional) histories. At that time I was a student of David G. Hays, one of the founders of CL, which is a rebranding of machine translation (MT). He was interested in how the human mind worked, which is why I was studying with him. He thought of AI as a bunch of unprincipled hackers. If the code worked, that's all. No need to think about the human mind.

Back to the Shakespeare model. I was interested in how people understand literary texts. At that time I was imagining a future when we could feed a text to a computer, then look under the hood, as it were, and see how the simulation "read" the text. Current LLMs can "read" literary texts, in some non-trivial sense of the term, but there's no reason to think they're doing it in the way humans. In particular, much of the emotion is the result of subcortical processing, but LLMs are all cortex if you will. Make no mistake, I'm glad to have them, but we've got a long way to go.

Expand full comment
4dEdited

I fully agree that whatever LLMs are doing isn't like what we're doing. And I like the phrasing you used here: "there's no reason to think" what they do bears resemblance to what we do. That's the argument that I find so compelling but which I think frustrates a lot of AI optimists. We don't know much about how our own mental states and abilities "work" in the physical sense. We don't know how to create life from non-life. Oh, there are theories! There are always theories. But I just see no good reason to believe, or even assign plausibility to, the notion that a talking computer built by humans is also the recipie for the special sauce from which intelligence "emerges". Can't prove it isn't, but why would it be? But, they want a positive refutation of their claims, one that can itself be falsified and in turn prove them right. They want "goalposts", and I don't think they're owed any.

I read the Aaronson and Pinker exchange, as well as the long comments thread below. It was a fascinating read. One thing I agree with Aaronson on is that claims which seem obviously true to some participants seem just as obviously false to others. When I read his arguments, and those of most people advocating for "computational theory of mind" (which I might not fully understand), I see routine conflation between our understanding of reality and reality itself. This is admittedly uncharitable, but they seem to have a gut sense that reality runs on math, and thus can be perfectly re-created, or at least simulated, using machines that run on math. And therefore in principle computers can have minds like ours. This just hits me as absurd - but to them denying it is absurd. Both sides are effectively accusing the other of believing in magic. That's interesting, if nothing else.

Expand full comment

Yes, the Aaronson/Pinker discussion is excellent, and the comments are worth reading through if you want to get a rich sense of what the argument is about.

Back in, I believe, 1976, Newell and Simon proposed the Physical Symbol System Hypothesis (PSSH) as the appropriate framework for understanding the mind. It was in the Turing Award paper, worth a read. More recently Saty Chary has proposed what he calls the Structured Physical System Hypothesis (SPSH), which I favor:

"‘A structured physical system has the necessary and sufficient means for specific intelligent response’. By structured physical system, I mean, an analog design, e.g. a Rube Goldberg apparatus, or a Braitenberg (!) vehicle, etc. This is in contrast to this: PSSH - Physical Symbol System Hypothesis - 'A physical symbol system has the necessary and sufficient means for general intelligent action'. "

https://new-savanna.blogspot.com/2022/08/structured-physical-system-hypothesis.html

Expand full comment

I suspect it's more that the people who talk the loudest about AI are not doing so to accurately inform the public. They're doing it solely to make sure that sweet sweet hype money keeps flowing. I find that the people doing the actual work on all these models are much less likely to say things like "in 5-10 years, AI will be able to solve poverty, cure cancer, prevent mental illness, stop corruption and bring about world peace." (Yes someone actually said that, and yes they are the CEO of a company with a multibillion dollar valuation).

Expand full comment

If you're referring to Dario Amodei's recent blog post "Machines of Loving Grace" (https://darioamodei.com/machines-of-loving-grace), he wasn't saying "in 5-10 years *from now* all these fantastic things will happen", he was saying "in 5-10 years from the point at which we develop powerful AI (which he then attempted to define) all these great things will happen". Still contentious, but not exactly what you're accusing him of.

Expand full comment

I'm sure it feels good that prominent other people are saying uncle. They aren't crying uncle yet so keep the pressure on.

Big tech growth has ended. Wily Coyote has reached maximum cliff overrun. Please keep showing LLM and image gen's funniest mistakes.

Expand full comment

I love it when he said, paraphrasing here: “You can perceive it as a wall, or you can perceive it as some small barriers.”

Expand full comment

Good AI (AGI-like qualities) is a red herring. GenAI doesn't need to be good to be disruptive.

I'm experiencing late 1990s déjà-vu

Expand full comment

True that GenAI doesn't need to be good. Thompson, the CEO of United Health pioneered the use of AI to deny claims. 90% "error" rate. Blame it on the AI. Not me!

Somehow, several other healthcare corporations figured out how to do the same thing.

Stunning, really.

A lot of pain and suffering, and a lot of deaths ensued from those denials.

Expand full comment

Including his, it would appear.

Expand full comment

OpenAI is about to release o1 pro. Ethan Mollick says it is very good.

There's lots of work ripe for automation, and lots of room for improvement in the AI offerings. AI companies should focus on near-term return rather than AGI.

Expand full comment

The psychology of writing on these issues is very illuminating. Gary recognises the signs that improvement will be harder and harder and writes about it (I tend to agree with Gary here, the basic architecture has fundamental limitations and no 'engineering the hell out of it' is going to fix that). You recognise positive statements about o1 and write about it. We all are signalling what supports our existing heartfelt convictions.

I do generally think statements like "about to release o1 pro and that is going to change everything" are somewhat premature (how often has 'going to be' been used in that way in AI hype? — often, including in serious magazines like New Scientist) and that one Ethan Mollick, who is a technology *policy* wonk, not a *technology* wonk is not that convincing. Then again, why believe Pichai? He has been spouting marketing/hype as well. Tomorrow it is the first anniversary of this one: https://blog.google/technology/ai/google-gemini-ai/#sundar-note

Yes, companies will focus on what can be done, not AGI (one hopes). But much of what people expect requires AGI-level performance. So, saying, let's not focus on AGI is fine, but then, recognise which uses effectively require it.

Aren't we having fun pushing our convictions?

Tomorrow too is the ARCprize announcement. My guess 😀: ~60% for the best performer this year.

Expand full comment

Best performer this year was 53.5%, ~2% *less* than the best performer last year.

Expand full comment

I don't think o1 is going to change everything.

The ARC prize is a very neat thing. It is forcing people to work on other kinds of modeling than language. We do need spatial reasoning, and many other things.

As to "basic architecture", it depends on what you mean. LLM can only distill and generate. There's huge value in that, but that's the equivalent of one reading a book. Useful, but that's where the real work is starting.

My bet is on an agent that can iteratively make hypotheses, invoke honest tools, do evaluations, and iterate. Language is a high-level medium of instruction, and I think it will play a big role.

I think language is more flexible than symbolic reasoning while retaining its ability for abstraction.

The focus is justifiably moving from distilling all one can in a pot, to making more use of what is distilled to do actual work.

Expand full comment

The misunderstanding afaic here is that 'language model' is a misleading label. These are not Large Language Models, they are Large Token Models (or Large Approximation Models). None of what happens with them has any relation with (humans) 'reading a book'.

Expand full comment

If you read a book in an area you know a lot about, then your mind automatically creates world models of things, and the book can help you improve your intuition.

If you read book on an unfamiliar topic, you get a sketchy idea of concepts, but you will have to work applying that knowledge.

The LLM is in the second mode. Its vast training can give it hints for how to proceed, but it needs careful observation of how it does, feedback, and corrections, until it learns to do well.

Expand full comment

You reading in English about an unfamiliar topic is not a good analogy (a.k.a. "lie to children"). Better is you reading a book on a familiar topic in an unknown language or coding.

E.g. this is you doing LLM's way of reading:

zyvcj fomoc caqtl ogush mvasf hkyvl famhl fqbrb niaap svzap cnund mgykt fobnp tmbqn vdfhk cghqn eeaby bytpj aukxc dgrnn pmccc hhgzb doorx xqpsj uuzir gqfkh lxgef mhmhh sphoh unjph shmyi wwfkx tfpyh nvnqo jexto vgioz hmawt nlheh lstfc ibmbz ylqgj pjaqx impdk xiftv oepjj odnfa psdgx emccy knqjk axvhq vftlr hufpe uznmg bhlnx uywzr eobym btogg wzfat rfnav bowyw fhfgu kknpq kvvjf xiiym qnxcp sznqk vukez igbax zbaqx lvaam ntpju txxzd wjhtz gqthk psnge fthz

The biggest 'lie to children' in all of this is conflating tokens with *linguistic* elements. They are as much linguistic as looking at text as a distribution of ink is. LLMs can approximate the results of understanding without having any.

Expand full comment

How is it in the "I told you so" club? Are the drinks as good as they say?

Expand full comment
6dEdited

The problem is that "deeper breakthroughs" cannot just be ordered up, no matter how much money is deployed. The next one is far more likely to be decades away than months away.

Expand full comment

Progress is likely going to be incremental. LLM was the quick shot in the arm by putting a lot of data in the pot and producing high-level estimates. The more detailed the work the longer it takes to get it right.

Expand full comment

It was important to keep things in perspective, 2 years ago, when the hype was at fever pitch.

It is important to keep things in perspective now. That may help one go through the "dark times".

The advancements in AI have been good. A tool that can do what AI chatbots can now would have been science fiction 5 years ago.

A lot more work is needed. We will keep on relying on large data and what we learned recently. The modeling needs to improve.

Expand full comment

Your post and this tweet came out less than a day apart:

@emollick: "It is worth noting that this [tweet below] is an increasingly common message from insiders at the big AI labs. It isn't unanimous, and you absolutely don't have to believe them, but I hear the same confidence privately as they are broadcasting publicly. (Still not sure how you plan for this)"

@OfficialLoganK: "If you are not planning for the price of intelligence to go to zero, the next 3-5 years are going to incredibly disruptive to your business / life."

https://x.com/emollick/status/1864509024214905060

Expand full comment
6dEdited

This sound like a Ray Kurzweil burner account. Variations on this type of prognostication have been with us for a very long time.

But the most "disruptive" result of AI, for most people, is that now when we search for things online we have to be on the lookout for AI generated websites and images and such. We have a new category of spam to try and filter out of our lives.

My bet is this will end up being the greatest enduring legacy of GenAI.

Expand full comment

Sam Altman willbe the last.

Expand full comment

Keep going, Gary. You're a light in this scene

Expand full comment

Marcus, you have me in total agreement with the interesting perspectives you have provided on Marcus on AI. As an investor (who cannot keep up with my benchmarks loaded with AI and AI derivative stocks), my frustration is if all this is true (which, I truly believe it is) and is being referenced by AI industry insiders, how come investors continue to reward AI stocks (and their management and BOD) to infinity. Why does Big Tech buy back their own stock at such a massive scale if they truly know the underlying AI technology they are spending massive capex on is not even close to matching expectations? I only see a few companies capable of monetizing their AI capex and I question the LT ROIC vs WACC given the levels of money they have thrown and continue to on AI. The price to sales multiples being paid are crazy! What is it going to take for the AI stock bubble to pop - a hyperscaler or two lowering AI capex guidance as a hint the ROIC is not meeting their cost of capital hurdle rates?? I recall a Bloomberg interview with Scott McNeally (Sun Microsystems CEO during the 95-00 Tech Bubble) in 2002 stating he could not believe investors were willing to pay a 10x revenue multiple for his company during the Tech Bubble. He stated he would have to conduct illegal activities in order to justify the economics of that multiple on his stock. Following is the link to the interview containing McNeally's comments on Tech valuations during the original Tech Bubble: https://www.bloomberg.com/news/articles/2002-03-31/a-talk-with-scott-mcnealy?sref=vuYGislZ

Expand full comment

Irrational exuberance.

Expand full comment

It continues to amaze me that one statistical process is called 'AI'." That should be a clue to the answer to Gary's fundamental question from his book Rebooting AI and paraphrased as "If AI is everything why can't it read." One answer is it can't read because reading is grounded in meaning. There is another AI I call a SAM (no not that Sam), a semantic AI model. This blog post explains the difference.

http://aicyc.org/2024/12/05/why-the-hype-around-emerging-agi-misses-the-mark-and-why-sam-is-the-solution/

TL;DR:The hype around LLMs achieving AGI is flawed because they rely on unstructured, often polluted data sources, leading to issues like misinformation and lack of verifiability. In contrast, SAM (Semantic AI Model) uses gold-standard, peer-reviewed sources to build a structured knowledge graph, ensuring grounded, traceable, and reliable information. SAM complements LLMs by providing clean, trustworthy knowledge, making it essential for achieving true AGI. Together, SAM and LLMs can combine scale, adaptability, and precision to tackle complex challenges, unlike LLMs alone.

Expand full comment

My money's on Google (or China) progressing to next stage

Expand full comment

If any of the current contenders would do it, it would indeed be DeepMind. Indeed, arguably they have already surpassed the others. They're the only one that appears to be thinking outside the box and from first principles.

Expand full comment

I tend to agree but Demis Hassabis puts too much faith in grounded representations delivering a breakthrough.

Expand full comment

I tend to agree but Demis Hassabis puts too much faith in grounded representations delivering a breakthrough.

Expand full comment