I agree that they are not the same thing. Learning an abstract representation of certain structures is more akin to useful memorization. Data leakage implies some sort of one-to-one correspondence between the input and output (that goes through the training, through embeddings and similar mechanisms), which sort of contradicts the nuanced or refined learning processes allegedly enabled by attention mechanisms. Creativity does require some degree of generalizability, but I would argue that, for various sorts of novel intermediate-level problems (which can be solved with 3 to 10 concrete steps, themselves achievable through iterative prompting or similar strategies), extrapolation (in the high-dimensional dataset space sense of large deep learning networks) might facilitate the discovery of useful solutions. My understanding is nascent and still evolving so please regard my comment as a perspective only.
In GenAI, as far as I know, 'memorisation' stands for regenerating source material (e.g. asking for 'that poem that compares a loved one to a summer's day' and get a perfect rendering of Shakespeare's Sonnet 18 as a result (or verbatim NYT articles in a more negative example). For creativity, see the other elements in this thread.
I must say that some of these reactions focus on how 'precisely correct' the statement is.
There are whiffs of 'bewitchment by language' here :-)
Uh, no. If neural networks can't refer to any actual concept at all, then how can they be creative _about_ anything, at all? How does anyone or anything be creative about nothing? Creativity must begin and end with referents. Where is the dangling referent to which creativity is focused upon?
The 'creativity' I mean comes from the slight randomness in pixel/token selection. I agree one could object to the use of the word 'creativity' here, but if you look for instance to the example I give for 'high temperature' GPT results in my talk, it seems that — however dumb the mechanism behind it — it is acceptable to label such results 'creative'.
However, it seems rather obvious to me that global temperature is not the answer, since actual creativity is *selective*.
(I.e. you don't want to be creative when evaluating 2+2, but fundamentally there really is no way to know (at runtime) whether the expression you are looking at requires creativity or rigour - I'd even conjecture that this precise function is NP-hard.)
OK, if you want to be *really* precise I would say that what we *experience* as creativity is the result of the 'constrained randomness' of the model's token/pixel selection. Discussing if that can be seen as 'being' creative gets us talking about meaning and use of words like 'creative'. In general, the use of 'creative' for this dumb process is acceptable. Throwing a dice many times 'creates' a random sequence, for instance.
Talking in detail about this being or not being correct use of the term 'creative' is I think a dead end.
Note that these GenAI's aren't bad at 'creativity' but above all at 'rigour' (because they have none).
You know we're not referring to a "dumb process" here. When you have a sentence such as "He is being creative" a "dumb process" isn't being referred to. Let's not pretend that some term being used doesn't mean what it means.
Medium length answer: If you create one you've solved AGI on digital computers and it will get you a Turing Award
Slightly longer answer: To be able to recognise a hallucination (note: everything from an LLM is technically a 'hallucination' (just stochastically constrained enough be approximate the results of understanding), the errors are not errors at all: https://ea.rna.nl/2023/11/01/the-hidden-meaning-of-the-errors-of-chatgpt-and-friends/) requires understanding at the level of language (LLMs 'understand' at the level of tokens). The relation between LLMs (slightly unpredictable token generation) and understanding is completely unsolved and nobody knows how to do it. For instance, to ask another system via plugin/action would require either the LLM to understand when to ask an what to ask, or it would require the system at the other end to full understand in the first place, but then you have to wonder why use an LLM in the first place.
Interesting points, and I think regarding refinements of hallucination, the refinement would be the LLM knows when it is hallucinating, so that it can employ it for creative purposes when appropriate.
As somebody who has been working computer security for as long as there was such a field, I can give you a rough idea of how bad the situation is. Think back to the basic elements of the WWII Ultra effort. There was one target, the Enigma cipher system. Breaking it gave its adversaries *everything.* Consider the amount of time and effort the adversaries put into achieving that break.
Now we have three targets: the OpenAI facility, the Gemini facility, and the Claude facility, plus the upcoming Stargate colossus. These are being constructed by organizations who have demonstrated no appreciation for the magnitude of the threats they face and no sympathy whatever for the direct and indirect costs required to respond to such a threat [1]. They are and will be the worst combination of soft target and (if they succeed in attracting enough business to be profitable) valuable target that has ever existed. Meditate on that and then consider the potential adversaries and examine the efforts those entities have mounted in the past in this area.
The true existential risk of GenAI is that it will succeed in being accepted, and by doing so will become essential. If its providers have not already been penetrated they soon will be, and that will be catastrophic for us in the way that Ultra was catastrophic for the Germans. Not through a single, massive event, but by operating at a constant disadvantage, one encounter after another, until ultimate defeat and collapse.
Any data these things use is scraped off the net so is largely irrelevant.
The only actually "useful" thing would be breaking in to like, find out someone has been searching for midget porn and blackmailing them. But that's already a thing that can be done.
"The true existential risk of GenAI is that it will succeed in being accepted, and by doing so will become essential."
oh yeah.
I feel like something of the sort has already happened to basic Google-type search capability, to a noticeable extent. Google c.2001, Mk I version, was just plain better with the clarity of its response to keyword prompts and its search ranking results, because it was unobstructed by The Market, and What Everybody Else Is Searching For, and Politics.
What's happened to Google resembles the "Best Match" search option that showed up on eBay one day out of nowhere some years ago, and has since been adopted by other marketplace platforms like Reverb. Speaking as someone doing the searches for the purpose of finding something to buy, Joe Consumer mode, the default of Best Match has never had the slightest utility for me. It's always been simply an irritant- an extra step that I need to take, in order to get rid of it and replace with a function that has some actual relevance, like ranking listings by price, or "most recently listed", or "ending soonest." It wasn't until a few weeks ago that I finally broke down and did an Internet keyword search to try to find out what "Best Match" mode is actually good for...surprise! It has nothing to do with facilitating searches by prospective Buyers; instead, it's an ad perk for eBay's preferred Sellers. (Full disclosure: I've been known to sell as well as buy on eBay and Reverb. At some times in the past, I've sold quite a lot. 5 Star Power Seller, all that. I still think Best Match is an example of a travesty of the way those search functions should work. Just like I think having a top page result full of ads and dumbed-down murkified crowdsourced search results is a travesty of a once near-miraculous Internet keyword search capability.)
In my opinion, a similar degradation has also taken place with the advent of "smart"phone ubiquity. The iPhone/Android and their variants and successors have become something like THE lingua franca computer device for the general public. But most of the capabilities found on actual full-size computers are severely hampered, because the screen and the touchscreen keyboard is too damn small to readily open multiple windows and do keyword searches in multiple windows, or to easily type out any communication with much more weight than perfunctory phatic exchanges punctuated with emojis, or the one-line canned snark that reduces story comment sections and platforms like Twitter to the level of middle school lunchroom banter. The trend is toward abridgement, and the information stovepiping that's the practically inevitable result of "subscriptions" and "apps."
When iPhones are taken for granted as having the same educational potential as computers, they don't broaden knowledge bases, they narrow them. (More disclosure: I own a big-screen iPhone.) IPhones emphasize the pictorial, the visual, moving pictures, A/V. Unsubtle eye-catching. It's an entertainment-advertising medium that fools waay too many people into thinking they're getting adequately educated and informed. And its superficial tropisms- nurtured by the massive feedback loop of the Used population- are increasingly being incorporated into the status quo of websites and pages designed for laptops and desktops. When catchy headlines and flashy accompanying pictures count more than story content, P. T. Barnum wins: "There's A Sucker Born Every Minute." "This Way To The Egress, $.05."
The AI output I've encountered will not improve this situation. To understate the case.
It's enough to get me to question Net Neutrality. Maybe that was part of the scheme all along; to eventually generate a "premium consumer" clientele begging to pay 3x-10x the rent-seeking base rate for their computer service, like an extra $1000-$2000 annually, to not have to page through server-farm volumes of midwit Troff'n'Brew fodder.
I'm prompted to recall that Before Time story, about a goose that laid golden eggs...
@Earl can you give us a scenario? What if the OpenAI facility is hacked. Then what? I'm trying to imagine what is as catastrophic as you are imagining. I am not a security expert but I did build a secrets management system at 1Password.
On point, as always. There’s without a doubt a growing asymmery between the size of the investments and the returns generated. Getting real business value from this technology, given the current state, is far from a walk in the park. In some ways it continues to be a solution looking for a problem.
I'm curious how much of that $3B in revenue comes from other AI companies spending money to make calls to GenAI APIs as their core business. Most of these companies are also making negligible amounts of revenue and propped up solely by the bubble.
Apart from the hustlers and frauds, what primarily drives the hype are the legions upon legions of people who rebut every criticism and every observation of a limitation with "it will only get better". There is this quasi-religious, ahistorical belief everywhere that things only ever get better, that there are no diminishing returns or structural constraints. And I don't see how that belief will ever go away; even in the face of a burst Gen AI bubble, they will just move on to the next thing, just as they moved on from blockchain and NFTs to Gen AI in the first place, because this belief system is cultish and part of their identity, and not evidence-based.
Yes, with the serious singularitarians, transhumanists, and so on it is really quite striking how they are merely plagiarising the ideas of religion, and mostly of Christianity, under different names:
Mind uploading only works if one sees the mind not as the process of the brain operating but as a spark that can be transferred from the body to somewhere else, i.e., a soul, but it makes absolutely no sense under materialist/physicalist assumptions.
Depending on the mood of the proponent, the singularity is either the rapture, with the anticipated superhuman, self-improving AI as a Messiah, or it is Ragnarok with the AI as a vengeful god destroying the world.
Roko's Basilisk re-invented eternal damnation.
Even the simulation hypothesis could be seen as a modern take on the idea that our perceived world is a mere shadow of True Forms that exist in a spiritual realm or in the mind of God.
But the main point here is the underlying attitude, the refusal to align one's beliefs with evidence as it is in front of us (just look at the actual quality of Gen AI outputs, or rather, the lack thereof!) in preference of toxic optimism and groupthink. That applies even to those many reply guys on social media who haven't imbibed all the accelerationist and singularitarian beliefs in detail.
Yes, the tendency is to believe that science is superseding religion through the application of reason and empiricism. Which to a degree it is. But there's also the part where it's actually displacing the priesthood with a new all -knowing class.
As does Microsoft Copilot, using RAG to look up up-to-date web based info, and it still very happily hallucinates and makes fundamental, confidence destroying mistakes, as Gary and I demonstrated a few weeks ago.
I was talking to an HR person who says they find it useful for the purpose of writing reports. They let an LLM write it then go back over the result and modify it as needed. They find this is considerably faster than starting from scratch.
I guess the point is that it can be useful but we should not expect miracles. However, who makes trillion dollar investment for anything short of a miracle?
I find it fascinating how capitalism actually blocks the development of AGI. As soon as there's MVP, in the form of GenAI, the whole market simply hypes that unrelentingly, and forgets all about investing the big bucks needed to overcome the actual problems of real intelligence. Fascinating to watch.
Well that is just in the short term. As soon as the hype dies away, attention on AGI may begin anew. Capitalism only blocks development in the way you are describing in order to reap short-term profits and facilitate wealth redistribution. In a way, the short-term evolutionary pressure is to maximize the benefit from the current iteration of technology, and then there WILL be space for AGI development.
It's all rather disgusting, and whether it happens now or later it will be a disaster as well.
I guess a lot will depend on just how far down the R&D road the next real yield point is. I have a suspicion that we actually know way less about the brain than we think we do. Gen AI may turn out to be a one hit wonder.
Why would you assume that AGI is even a useful goal?
AGI is pretty much either A) a religious hallucination or B) kind of worthless.
We already have general intelligences - they're called humans. Improving those via genetic engineering holds far more promise than machine learning does for generating a better intelligence.
The goal of automated systems is not to create intelligence but efficiency. You don't want your art AI to be smart, you want it to do what it is told.
AGI is pretty worthless in this regard (who wants a robot slave?), but generative AI is way more useful. You don't want intelligent robots, you want dumb ones that operate safely and efficiently.
"Thus far, the closest thing to a killer app is probably coding assistace, but the income there isn’t massive enough to cover costs of the chips, legal etc."
I was writing out this very sentence in my mind before reading your paragraph.
Now, this is not to brag about my own anticipation of your point (only *very* tangentially), but rather to reinforce that there is actually no money AT ALL in coding assistance:
Downloading starcoder (or something similar) from Huggingface takes literally minutes and depending on how much UI you need, deployment is another 30 minutes or so. If you have a GPU with 8+ GB of VRAM - and chances are if you are an *actual* coder, you have *more* VRAM than that - it eats 80 Watts (or so) when generating, but it won't generate all the time, so the cost is substantially less than 80 Wh*200*8 = how much is ~128 kWh in the US? 20$ or so? (You have the GPU anyways, so don't factor it in...)
No, coding is not profitable, and that's not even counting in that these things make costly mistakes (*duh*, I know...).
NFTs and the blockchain were always blatantly useless. Generative AI, on the other hand, has produced deliverables. Image gen in particular has produced an insane number of images and the quality has gone up extremely far, extremely fast, to the point where it can be used to produce art that is sufficient quality to illustrate indie products and games and whatnot.
If you think generative AI is at all similar to these, you're not really living in reality. Crypto is a classic con. MidJourney is a business that delivers a product that people actually use.
Text LLMs are a toy for now, but image gen is already valuable.
It may feel like it, but it already has many more practical applications (or societal disruptions, if you prefer) than crypto. Unlike crypto and blockchain, actual people are being replaced with AI. As to how big the hype really is, well that is a question more interesting to investors than sociologists.
Loved this article. As a builder I have a lot of the same concerns. Many people are less euphoric as they experience the reality of the models short comings and insane costs as the jam more and more tokens into their prompts as their product evolves and usage grows. Yet I am not yet seeing people throw their hands up in the air. Yes the economics have to sort out, but I still believe that if you don't develop a corporate expertise and a personal expertise in AI and start now you will get left behind, and I am talking about being left so far off of the back that it hurts - BIG TIME - no job, competitors out selling you and achieving a lower cost structure and so on.
Total deals are dropping but total amount invested is somewhat flat since the hype started. This may be that venture has to deploy its capital and this is still the most interesting place, even with the risks.
Overall good article but I would temper the article a bit. Glad I could help!
While I realize that, taken directly, the following is a false equivalency, but this entire LLM fascination reminds me of string theory, but with even more money sunk in. Great hopes that this will be the "unifying" foundation of AGI, yet with scant evidence thus far to back up that assertion; just a lot of pretty equations and clever parlor tricks.
I think people need to just delete the pseudo-religious term AGI from their vocabulary, honestly.
Automation is about producing things faster, better, and more efficiently. Image gen is super fast and produces pretty good quality - but it is vastly cheaper than commissioning bespoke art, and vastly faster and more accessible to the public.
That's the real value, not "AGI". Why would you want an intelligent robot? You wouldn't. That's just a slave.
What you want is a highly efficient machine that accurately does what you tell it to do.
Generative AI, and neural networks in general, don’t deal with referents, and they never truly will. Try asking it to give you a picture of a room with NO elephants in it and see what happens. They don’t deal with concepts. They pretend to, like all machines can only do. https://towardsdatascience.com/artificial-consciousness-is-impossible-c1b2ab0bdc46
Not sure why you think this is impossible for these AIs to do. Indeed, people have been playing around with "negative space" to see what happens if you tell it NOT to make a bunch of things for like two years now; it's quite interesting. You can actually tell them to exclude things.
It isn't intelligent, but it is capable of that. All it is, ultimately, is a statistical weighting algorithm. Telling it not to include something means it actively excludes things like that thing.
Well, exact what prompt did you use because when I clicked I got nothing but a 403 server error. Also, I didn’t say impossible… that’s not what I said. I did say these systems don’t handle concepts, and your prompt may well serve to make my point.
Cooked prompt. Not using English grammar. Why "digital painting?" Try "Create a picture of an empty room with no elephants" ...another demonstration would be the concept of "ouroboros". It doesn't have an actual concept of what an ouroboros is. The Webster dictionary definition is that of "a circular symbol that depicts a snake or dragon devouring its own tail" I ask it to just draw me an ouroboros and it keeps messing it up.
The bot did exactly what I told it to do using very simple instructions. The fact that I didn't use "English grammar" is irrelevant. A bot not speaking English doesn't mean it can't take instructions.
You didn't think that the bot could do that, or would function in that way, so are looking for reasons why you are right, rather than accepting that you are wrong and changing to the correct position.
It isn't intelligent but that doesn't mean it isn't useful or functional.
It's definitely not capable of producing literally anything, but that's true of anything, including people.
I was with you until your last sentence. I see no reason knowledge and understanding can't be computed. LLMs are bad at it for reasons we completely understand. On the other hand, a thermostat "knows" the current temperature and knows what to do about it. Everything else is a matter of understanding what brains do and applying it. Statistical analysis of large amounts of text is useful, but solves the wrong problem.
If it doesn't deal with referents then how is anything "understood"? Also, "understanding what brains do" is a bit of a fallacious presumption. There's not going to be a "correct modeling," period. It's underdetermined. https://plato.stanford.edu/entries/scientific-underdetermination/
Knowing something is reflected in the behavior of the agent in question. If the thermostat makes decisions based on the current temperature, it knows the temperature. If it is reflected in its behavior, then the agent knows the bit of knowledge in question. The internals matter, of course, if we want to understand how the agent's behavior is produced but there is no promise that we will easily understand how a particular bit of knowledge is represented within a complex agent.
It's not behaviorism just because the word "behavior" appears in it. Behaviorism, at least as Skinner had it, was just a reaction to the time and was silly, IMHO. As far as the Chinese Room Argument, a lot has been said already about that. It's just wrong. It has so many goofy elements that it fails completely as a thought experiment. Searle either didn't understand computers or was willing to sacrifice his professional reputation for the sake of claiming humans to be intrinsically superior to AI. That's not a position I have any respect for.
Since you are not really coming to grips with what I said in my comment, lets end it here. I have no interest in having a Chinese Room Argument.
Uh, that's not a counterargument. "Searle is wrong and what he said was goofy" isn't a counterargument. Do you even know how to make a point? You're arguing by assertion- Try actually addressing Searle's argument, starting with how the Chinese Room doesn't actually understand Chinese. If you disagree, try explaining how the room actually does.
As long as NN is there it's going to be a kludge. NNs don't belong in a system involving any kind of legitimate epistemology; Their presence de-legitimizes knowledge claims.
NNs as a technology is a distraction that's holding back AI development as a whole
To those who've drunk the kool-aid, nothing will convince them otherwise. Any student of Morovec's Paradox, should by now see that LLMs, indeed GenAI as a model, lacks the foundations needed to distinguish nonsense from reality. The root of the problem is that GenAI never learns how to reason at all; it just scrapes up selected patterns of reasoning that are already present in the vast corpus of predigested human thought provided in its training set. It has no introspection, no ongoing interactions with a lived reality that every human uses to decide if something is objectively true, or just fantasy. GenAIs are the expert systems of this new century; ask Doug Lenat how that worked out.
What still isn't clear to most people is that with GenAI
(1) useful memorisation and unacceptable training data leakage are technically the same thing
(2) creativity and 'hallucinations' are technically the same thing
They are the same. We just stick two different labels on them based on if we like/want the result or not
totally not the same thing
1) useful memorisation refers to memorizing the essence of something, not its verbatim expression
2) creativity requires generalization, genAI isn't capable of that, it's easily demonstrated with the inability to learn simple arithmetic
Gerben Wierda talks about how it is, and you talk about how it should be :-)
I agree that they are not the same thing. Learning an abstract representation of certain structures is more akin to useful memorization. Data leakage implies some sort of one-to-one correspondence between the input and output (that goes through the training, through embeddings and similar mechanisms), which sort of contradicts the nuanced or refined learning processes allegedly enabled by attention mechanisms. Creativity does require some degree of generalizability, but I would argue that, for various sorts of novel intermediate-level problems (which can be solved with 3 to 10 concrete steps, themselves achievable through iterative prompting or similar strategies), extrapolation (in the high-dimensional dataset space sense of large deep learning networks) might facilitate the discovery of useful solutions. My understanding is nascent and still evolving so please regard my comment as a perspective only.
In GenAI, as far as I know, 'memorisation' stands for regenerating source material (e.g. asking for 'that poem that compares a loved one to a summer's day' and get a perfect rendering of Shakespeare's Sonnet 18 as a result (or verbatim NYT articles in a more negative example). For creativity, see the other elements in this thread.
I must say that some of these reactions focus on how 'precisely correct' the statement is.
There are whiffs of 'bewitchment by language' here :-)
Uh, no. If neural networks can't refer to any actual concept at all, then how can they be creative _about_ anything, at all? How does anyone or anything be creative about nothing? Creativity must begin and end with referents. Where is the dangling referent to which creativity is focused upon?
The 'creativity' I mean comes from the slight randomness in pixel/token selection. I agree one could object to the use of the word 'creativity' here, but if you look for instance to the example I give for 'high temperature' GPT results in my talk, it seems that — however dumb the mechanism behind it — it is acceptable to label such results 'creative'.
No. A six-sided die you throw is not creative. It's a stupid die.
I think what Gerben is trying to say is: yes, it's not creative, it's rolling a die.
But if you're rolling enough dice, then it GIVES THE APPEARANCE of creativity.
Yes, and I think I understand the sentiment.
However, it seems rather obvious to me that global temperature is not the answer, since actual creativity is *selective*.
(I.e. you don't want to be creative when evaluating 2+2, but fundamentally there really is no way to know (at runtime) whether the expression you are looking at requires creativity or rigour - I'd even conjecture that this precise function is NP-hard.)
OK, if you want to be *really* precise I would say that what we *experience* as creativity is the result of the 'constrained randomness' of the model's token/pixel selection. Discussing if that can be seen as 'being' creative gets us talking about meaning and use of words like 'creative'. In general, the use of 'creative' for this dumb process is acceptable. Throwing a dice many times 'creates' a random sequence, for instance.
Talking in detail about this being or not being correct use of the term 'creative' is I think a dead end.
Note that these GenAI's aren't bad at 'creativity' but above all at 'rigour' (because they have none).
You know we're not referring to a "dumb process" here. When you have a sentence such as "He is being creative" a "dumb process" isn't being referred to. Let's not pretend that some term being used doesn't mean what it means.
Is there a plug to identify whether a creativity is actually a hallucination?
Short answer: no.
Medium length answer: If you create one you've solved AGI on digital computers and it will get you a Turing Award
Slightly longer answer: To be able to recognise a hallucination (note: everything from an LLM is technically a 'hallucination' (just stochastically constrained enough be approximate the results of understanding), the errors are not errors at all: https://ea.rna.nl/2023/11/01/the-hidden-meaning-of-the-errors-of-chatgpt-and-friends/) requires understanding at the level of language (LLMs 'understand' at the level of tokens). The relation between LLMs (slightly unpredictable token generation) and understanding is completely unsolved and nobody knows how to do it. For instance, to ask another system via plugin/action would require either the LLM to understand when to ask an what to ask, or it would require the system at the other end to full understand in the first place, but then you have to wonder why use an LLM in the first place.
Yes! They are screwed either way. Get it right and it's copyright infringement. Get it wrong and, well, no one wants wrong.
Interesting points, and I think regarding refinements of hallucination, the refinement would be the LLM knows when it is hallucinating, so that it can employ it for creative purposes when appropriate.
"the refinement would be the LLM knows when it is hallucinating"
the achievement of that capability sounds to me as if it would be more than a "refinement".
As somebody who has been working computer security for as long as there was such a field, I can give you a rough idea of how bad the situation is. Think back to the basic elements of the WWII Ultra effort. There was one target, the Enigma cipher system. Breaking it gave its adversaries *everything.* Consider the amount of time and effort the adversaries put into achieving that break.
Now we have three targets: the OpenAI facility, the Gemini facility, and the Claude facility, plus the upcoming Stargate colossus. These are being constructed by organizations who have demonstrated no appreciation for the magnitude of the threats they face and no sympathy whatever for the direct and indirect costs required to respond to such a threat [1]. They are and will be the worst combination of soft target and (if they succeed in attracting enough business to be profitable) valuable target that has ever existed. Meditate on that and then consider the potential adversaries and examine the efforts those entities have mounted in the past in this area.
The true existential risk of GenAI is that it will succeed in being accepted, and by doing so will become essential. If its providers have not already been penetrated they soon will be, and that will be catastrophic for us in the way that Ultra was catastrophic for the Germans. Not through a single, massive event, but by operating at a constant disadvantage, one encounter after another, until ultimate defeat and collapse.
1. See: https://arstechnica.com/security/2024/03/thousands-of-servers-hacked-in-ongoing-attack-targeting-ray-ai-framework/
Terrifying and well said
Any data these things use is scraped off the net so is largely irrelevant.
The only actually "useful" thing would be breaking in to like, find out someone has been searching for midget porn and blackmailing them. But that's already a thing that can be done.
What does AI change? Nothing.
"The true existential risk of GenAI is that it will succeed in being accepted, and by doing so will become essential."
oh yeah.
I feel like something of the sort has already happened to basic Google-type search capability, to a noticeable extent. Google c.2001, Mk I version, was just plain better with the clarity of its response to keyword prompts and its search ranking results, because it was unobstructed by The Market, and What Everybody Else Is Searching For, and Politics.
What's happened to Google resembles the "Best Match" search option that showed up on eBay one day out of nowhere some years ago, and has since been adopted by other marketplace platforms like Reverb. Speaking as someone doing the searches for the purpose of finding something to buy, Joe Consumer mode, the default of Best Match has never had the slightest utility for me. It's always been simply an irritant- an extra step that I need to take, in order to get rid of it and replace with a function that has some actual relevance, like ranking listings by price, or "most recently listed", or "ending soonest." It wasn't until a few weeks ago that I finally broke down and did an Internet keyword search to try to find out what "Best Match" mode is actually good for...surprise! It has nothing to do with facilitating searches by prospective Buyers; instead, it's an ad perk for eBay's preferred Sellers. (Full disclosure: I've been known to sell as well as buy on eBay and Reverb. At some times in the past, I've sold quite a lot. 5 Star Power Seller, all that. I still think Best Match is an example of a travesty of the way those search functions should work. Just like I think having a top page result full of ads and dumbed-down murkified crowdsourced search results is a travesty of a once near-miraculous Internet keyword search capability.)
In my opinion, a similar degradation has also taken place with the advent of "smart"phone ubiquity. The iPhone/Android and their variants and successors have become something like THE lingua franca computer device for the general public. But most of the capabilities found on actual full-size computers are severely hampered, because the screen and the touchscreen keyboard is too damn small to readily open multiple windows and do keyword searches in multiple windows, or to easily type out any communication with much more weight than perfunctory phatic exchanges punctuated with emojis, or the one-line canned snark that reduces story comment sections and platforms like Twitter to the level of middle school lunchroom banter. The trend is toward abridgement, and the information stovepiping that's the practically inevitable result of "subscriptions" and "apps."
When iPhones are taken for granted as having the same educational potential as computers, they don't broaden knowledge bases, they narrow them. (More disclosure: I own a big-screen iPhone.) IPhones emphasize the pictorial, the visual, moving pictures, A/V. Unsubtle eye-catching. It's an entertainment-advertising medium that fools waay too many people into thinking they're getting adequately educated and informed. And its superficial tropisms- nurtured by the massive feedback loop of the Used population- are increasingly being incorporated into the status quo of websites and pages designed for laptops and desktops. When catchy headlines and flashy accompanying pictures count more than story content, P. T. Barnum wins: "There's A Sucker Born Every Minute." "This Way To The Egress, $.05."
The AI output I've encountered will not improve this situation. To understate the case.
It's enough to get me to question Net Neutrality. Maybe that was part of the scheme all along; to eventually generate a "premium consumer" clientele begging to pay 3x-10x the rent-seeking base rate for their computer service, like an extra $1000-$2000 annually, to not have to page through server-farm volumes of midwit Troff'n'Brew fodder.
I'm prompted to recall that Before Time story, about a goose that laid golden eggs...
@Earl can you give us a scenario? What if the OpenAI facility is hacked. Then what? I'm trying to imagine what is as catastrophic as you are imagining. I am not a security expert but I did build a secrets management system at 1Password.
You should re-post that as a reply to Earl. I don't know
On point, as always. There’s without a doubt a growing asymmery between the size of the investments and the returns generated. Getting real business value from this technology, given the current state, is far from a walk in the park. In some ways it continues to be a solution looking for a problem.
I'm curious how much of that $3B in revenue comes from other AI companies spending money to make calls to GenAI APIs as their core business. Most of these companies are also making negligible amounts of revenue and propped up solely by the bubble.
Apart from the hustlers and frauds, what primarily drives the hype are the legions upon legions of people who rebut every criticism and every observation of a limitation with "it will only get better". There is this quasi-religious, ahistorical belief everywhere that things only ever get better, that there are no diminishing returns or structural constraints. And I don't see how that belief will ever go away; even in the face of a burst Gen AI bubble, they will just move on to the next thing, just as they moved on from blockchain and NFTs to Gen AI in the first place, because this belief system is cultish and part of their identity, and not evidence-based.
Yeah, the West is seriously missing religion, it seems clear.
Yes, with the serious singularitarians, transhumanists, and so on it is really quite striking how they are merely plagiarising the ideas of religion, and mostly of Christianity, under different names:
Mind uploading only works if one sees the mind not as the process of the brain operating but as a spark that can be transferred from the body to somewhere else, i.e., a soul, but it makes absolutely no sense under materialist/physicalist assumptions.
Depending on the mood of the proponent, the singularity is either the rapture, with the anticipated superhuman, self-improving AI as a Messiah, or it is Ragnarok with the AI as a vengeful god destroying the world.
Roko's Basilisk re-invented eternal damnation.
Even the simulation hypothesis could be seen as a modern take on the idea that our perceived world is a mere shadow of True Forms that exist in a spiritual realm or in the mind of God.
But the main point here is the underlying attitude, the refusal to align one's beliefs with evidence as it is in front of us (just look at the actual quality of Gen AI outputs, or rather, the lack thereof!) in preference of toxic optimism and groupthink. That applies even to those many reply guys on social media who haven't imbibed all the accelerationist and singularitarian beliefs in detail.
I used to called the singularity Nerd Rapture
Alex SL *was* pointing to a religion! The (Western) religions of materialism, neuroscience, technology, the Singularity, AI, etc.
Yes, the tendency is to believe that science is superseding religion through the application of reason and empiricism. Which to a degree it is. But there's also the part where it's actually displacing the priesthood with a new all -knowing class.
Even Perplexity's "answer engine" which DOES have capability to look up the most current info it can still hallucinates.
As does Microsoft Copilot, using RAG to look up up-to-date web based info, and it still very happily hallucinates and makes fundamental, confidence destroying mistakes, as Gary and I demonstrated a few weeks ago.
I was talking to an HR person who says they find it useful for the purpose of writing reports. They let an LLM write it then go back over the result and modify it as needed. They find this is considerably faster than starting from scratch.
I guess the point is that it can be useful but we should not expect miracles. However, who makes trillion dollar investment for anything short of a miracle?
Or was that 100T?
Reminds me of the “blockchain-all-the-things” hype of 2016-2020, except the GenAI demos are way cooler 😆
Generative AI is actually useful.
Blockchains are just distributed ledgers and have always been useless.
I find it fascinating how capitalism actually blocks the development of AGI. As soon as there's MVP, in the form of GenAI, the whole market simply hypes that unrelentingly, and forgets all about investing the big bucks needed to overcome the actual problems of real intelligence. Fascinating to watch.
Precisely
Well that is just in the short term. As soon as the hype dies away, attention on AGI may begin anew. Capitalism only blocks development in the way you are describing in order to reap short-term profits and facilitate wealth redistribution. In a way, the short-term evolutionary pressure is to maximize the benefit from the current iteration of technology, and then there WILL be space for AGI development.
It's all rather disgusting, and whether it happens now or later it will be a disaster as well.
I guess a lot will depend on just how far down the R&D road the next real yield point is. I have a suspicion that we actually know way less about the brain than we think we do. Gen AI may turn out to be a one hit wonder.
Why would you assume that AGI is even a useful goal?
AGI is pretty much either A) a religious hallucination or B) kind of worthless.
We already have general intelligences - they're called humans. Improving those via genetic engineering holds far more promise than machine learning does for generating a better intelligence.
The goal of automated systems is not to create intelligence but efficiency. You don't want your art AI to be smart, you want it to do what it is told.
AGI is pretty worthless in this regard (who wants a robot slave?), but generative AI is way more useful. You don't want intelligent robots, you want dumb ones that operate safely and efficiently.
"Thus far, the closest thing to a killer app is probably coding assistace, but the income there isn’t massive enough to cover costs of the chips, legal etc."
I was writing out this very sentence in my mind before reading your paragraph.
Now, this is not to brag about my own anticipation of your point (only *very* tangentially), but rather to reinforce that there is actually no money AT ALL in coding assistance:
Downloading starcoder (or something similar) from Huggingface takes literally minutes and depending on how much UI you need, deployment is another 30 minutes or so. If you have a GPU with 8+ GB of VRAM - and chances are if you are an *actual* coder, you have *more* VRAM than that - it eats 80 Watts (or so) when generating, but it won't generate all the time, so the cost is substantially less than 80 Wh*200*8 = how much is ~128 kWh in the US? 20$ or so? (You have the GPU anyways, so don't factor it in...)
No, coding is not profitable, and that's not even counting in that these things make costly mistakes (*duh*, I know...).
Was just recently thinking about how the GenAI hype feels similar to the hype around crypto a few years ago. I'm glad that example was brought up.
NFTs and the blockchain were always blatantly useless. Generative AI, on the other hand, has produced deliverables. Image gen in particular has produced an insane number of images and the quality has gone up extremely far, extremely fast, to the point where it can be used to produce art that is sufficient quality to illustrate indie products and games and whatnot.
If you think generative AI is at all similar to these, you're not really living in reality. Crypto is a classic con. MidJourney is a business that delivers a product that people actually use.
Text LLMs are a toy for now, but image gen is already valuable.
It may feel like it, but it already has many more practical applications (or societal disruptions, if you prefer) than crypto. Unlike crypto and blockchain, actual people are being replaced with AI. As to how big the hype really is, well that is a question more interesting to investors than sociologists.
Loved this article. As a builder I have a lot of the same concerns. Many people are less euphoric as they experience the reality of the models short comings and insane costs as the jam more and more tokens into their prompts as their product evolves and usage grows. Yet I am not yet seeing people throw their hands up in the air. Yes the economics have to sort out, but I still believe that if you don't develop a corporate expertise and a personal expertise in AI and start now you will get left behind, and I am talking about being left so far off of the back that it hurts - BIG TIME - no job, competitors out selling you and achieving a lower cost structure and so on.
I was guessing that venture would have backed out more than they have. https://news.crunchbase.com/venture/monthly-global-funding-recap-february-2024/
Total deals are dropping but total amount invested is somewhat flat since the hype started. This may be that venture has to deploy its capital and this is still the most interesting place, even with the risks.
Overall good article but I would temper the article a bit. Glad I could help!
While I realize that, taken directly, the following is a false equivalency, but this entire LLM fascination reminds me of string theory, but with even more money sunk in. Great hopes that this will be the "unifying" foundation of AGI, yet with scant evidence thus far to back up that assertion; just a lot of pretty equations and clever parlor tricks.
I think people need to just delete the pseudo-religious term AGI from their vocabulary, honestly.
Automation is about producing things faster, better, and more efficiently. Image gen is super fast and produces pretty good quality - but it is vastly cheaper than commissioning bespoke art, and vastly faster and more accessible to the public.
That's the real value, not "AGI". Why would you want an intelligent robot? You wouldn't. That's just a slave.
What you want is a highly efficient machine that accurately does what you tell it to do.
Generative AI, and neural networks in general, don’t deal with referents, and they never truly will. Try asking it to give you a picture of a room with NO elephants in it and see what happens. They don’t deal with concepts. They pretend to, like all machines can only do. https://towardsdatascience.com/artificial-consciousness-is-impossible-c1b2ab0bdc46
I just told NijiJourney V6 to make me a digital painting of a room with no elephants.
And it gave me...
https://www.deviantart.com/stash/021g1zlbu5w8
A digital painting of a room with no elephants.
Not sure why you think this is impossible for these AIs to do. Indeed, people have been playing around with "negative space" to see what happens if you tell it NOT to make a bunch of things for like two years now; it's quite interesting. You can actually tell them to exclude things.
It isn't intelligent, but it is capable of that. All it is, ultimately, is a statistical weighting algorithm. Telling it not to include something means it actively excludes things like that thing.
Well, exact what prompt did you use because when I clicked I got nothing but a 403 server error. Also, I didn’t say impossible… that’s not what I said. I did say these systems don’t handle concepts, and your prompt may well serve to make my point.
https://drive.google.com/file/d/1wvhxAscK_NhlM7z8quh9nMplG_vFHt3g/view?usp=sharing
It's literally just "Digital painting of an empty room --no elephants --niji 6".
Cooked prompt. Not using English grammar. Why "digital painting?" Try "Create a picture of an empty room with no elephants" ...another demonstration would be the concept of "ouroboros". It doesn't have an actual concept of what an ouroboros is. The Webster dictionary definition is that of "a circular symbol that depicts a snake or dragon devouring its own tail" I ask it to just draw me an ouroboros and it keeps messing it up.
The bot did exactly what I told it to do using very simple instructions. The fact that I didn't use "English grammar" is irrelevant. A bot not speaking English doesn't mean it can't take instructions.
You didn't think that the bot could do that, or would function in that way, so are looking for reasons why you are right, rather than accepting that you are wrong and changing to the correct position.
It isn't intelligent but that doesn't mean it isn't useful or functional.
It's definitely not capable of producing literally anything, but that's true of anything, including people.
So did you try ouroboros or not
“Correct position”… that’s just laughable… you’ll see why sooner or later
I was with you until your last sentence. I see no reason knowledge and understanding can't be computed. LLMs are bad at it for reasons we completely understand. On the other hand, a thermostat "knows" the current temperature and knows what to do about it. Everything else is a matter of understanding what brains do and applying it. Statistical analysis of large amounts of text is useful, but solves the wrong problem.
If it doesn't deal with referents then how is anything "understood"? Also, "understanding what brains do" is a bit of a fallacious presumption. There's not going to be a "correct modeling," period. It's underdetermined. https://plato.stanford.edu/entries/scientific-underdetermination/
Knowing something is reflected in the behavior of the agent in question. If the thermostat makes decisions based on the current temperature, it knows the temperature. If it is reflected in its behavior, then the agent knows the bit of knowledge in question. The internals matter, of course, if we want to understand how the agent's behavior is produced but there is no promise that we will easily understand how a particular bit of knowledge is represented within a complex agent.
I have a handy three-word reply: Chinese Room Argument. Searle had already demonstrated decades ago how behaviorism doesn't fly.
It's not behaviorism just because the word "behavior" appears in it. Behaviorism, at least as Skinner had it, was just a reaction to the time and was silly, IMHO. As far as the Chinese Room Argument, a lot has been said already about that. It's just wrong. It has so many goofy elements that it fails completely as a thought experiment. Searle either didn't understand computers or was willing to sacrifice his professional reputation for the sake of claiming humans to be intrinsically superior to AI. That's not a position I have any respect for.
Since you are not really coming to grips with what I said in my comment, lets end it here. I have no interest in having a Chinese Room Argument.
Uh, that's not a counterargument. "Searle is wrong and what he said was goofy" isn't a counterargument. Do you even know how to make a point? You're arguing by assertion- Try actually addressing Searle's argument, starting with how the Chinese Room doesn't actually understand Chinese. If you disagree, try explaining how the room actually does.
The baseline referent of autonomous individual human intelligence is Corporeality.
if a machine has no Corporeality, then what kind of autonomous referent to Reality does it have?
Those are ultimately kludges and bandaids. It will be a perpetual band-aiding exercise. See this famous "Panda" example of how NNs don't actually identify objects mentioned in this article https://towardsdatascience.com/fooling-neural-networks-with-adversarial-examples-8afd36258a03
As long as NN is there it's going to be a kludge. NNs don't belong in a system involving any kind of legitimate epistemology; Their presence de-legitimizes knowledge claims.
NNs as a technology is a distraction that's holding back AI development as a whole
Sunken cost fallacy. No, I'm not the one standing on the wrong side of "ideology" here. https://aeon.co/essays/your-brain-does-not-process-information-and-it-is-not-a-computer https://www.theguardian.com/science/2020/feb/27/why-your-brain-is-not-a-computer-neuroscience-neural-networks-consciousness
To those who've drunk the kool-aid, nothing will convince them otherwise. Any student of Morovec's Paradox, should by now see that LLMs, indeed GenAI as a model, lacks the foundations needed to distinguish nonsense from reality. The root of the problem is that GenAI never learns how to reason at all; it just scrapes up selected patterns of reasoning that are already present in the vast corpus of predigested human thought provided in its training set. It has no introspection, no ongoing interactions with a lived reality that every human uses to decide if something is objectively true, or just fantasy. GenAIs are the expert systems of this new century; ask Doug Lenat how that worked out.
Reminds me of the “AI Winter” of the mid 1980s.