The suffocating hype is a major source of my skepticism. Stop telling me how great it is, and start giving me concrete and measurable examples. Not examples that fall apart with the tiniest bit of scrutiny. The Information had a post a few days ago about using it in sales and instructing it to insert typos so that customers wouldn’t realize it’s AI generated. Tricking people is not how we build trust, and trust is the foundation of so many good things.
That about the suffocating naysayers, doomers, nattering nabobs saying nothing is working, AI sucks, etc.? Interesting that these people want to have it both ways. AI is not ready for primetime! But we really really need to worry about the existential threat from AI....
The naysayers and the doomers are definitely not in the same camp. Speaking as a proud naysayer, I think AI doomers are the most outrageous hypesters of them all. Sam Altman's got nothing on Eliezer Yudkowsky when it comes to irresponsible, pseudo-religious prognistication.
I see the advertisements that read along the lines of, "Use AI to provide a personal response." I shake my head that they've not only twisted the definition of "intelligence", but they're twisting the definition of "personal". Yes, anything that is truly "AI" should not be applied as a trick.
I'm also seeing more shrill "AI-good-cause-washing", where genAI is being pushed as the solution to society's various ills. Eric Schmidt's "AI will solve climate crisis" is perhaps the most absurd and high profile example, but I attended a healthcare conference last week where nVidia were pushing their foundation model insta-solutions to the various challenges in the drug discovery industry. This was met by much weary eye rolling from executives for whom this isn't their first such rodeo, but perhaps plays better in the general media. I take all this as signs that things are slowing down worryingly for the world's largest tech companies.
Same! Don't know her personally but it seemed way out of left field. I wondered whether she had really written it or whether she's got a team posting on her behalf. In which case... the end result is the same.
I also think that Kazyrkov got it backwards. I would say this:
*The most powerful people in the world have been bamboozled by AI hype.
*They over-estimate AI's capabilities (at least the current state of the art).
*They over-estimate the value to their business and under-estimate the work required to receive value.
*They think that they understand it, but do not. The current crop of models are LANGUAGE models, not cognitive models, not reasoning models.
*They think that they know how to use it. Co-pilots can be useful, but they are only a tiny fraction of what the current AI models can do.
*They think that an AI product is one that simply repackages chatbots.
*They have no idea of the critical role played by data in all of these AI models. They under-estimated the cost of getting good data and over-estimate the accuracy of the system.
*Some of these powerful people fear what they guess AI will do and they seek to regulate it. But, being fooled by the hype, they risk focusing the regulations on the wrong problems, and missing the critical targets.
“OpenAI has released a new benchmark test to assess the factual accuracy of AI models and found their own lacking. It ran its own AI models through the test and found that its best-performing model–GPT-o1—scored just 42.7%, with its next best model—GPT-4-o—scoring a measly 38.2%. Its smallest model—GPT-4o mini—only scored 8%, but OpenAI has blamed this on its small size, saying it has ‘little knowledge about the world'. Although the GPT scores are poor, Anthropic's best model, Claude-3.5 Sonnet, only got 28.9% of questions right, underscoring the worrying factual accuracy of AI models, and perhaps highlighting that they shouldn't be used as a single source of information.”
Also, based on conducting a pilot in my organization, I have found that it is impossible for most people to justify the $30/month cost. The productivity gains are very limited with Microsoft co-pilot. In some cases, people spent more time verifying the information produced by the copilots rather than performing the activity without one. In other cases, some people blindly trusted the information.
Integrating copilot or GenAI into enterprise applications is risky unless you can trust the information produced by these products. We are not there yet, and we may not be for a while or ever with current models.
It is probably no coincidence that 'influencer' and 'influenza' are such similar words. The things they connotate are both to be avoided if you value your health.
While he's a Microsoft competitor, Ben Thompson's interview with Marc Benioff outlined the limits and applications of LLMs. As Salesforce's CEO, Benioff is actively integrating LLM technology into Salesforce's products.
With that, a couple of MB punchlines. My favorite is, "there's two places customers have been dramatically misled, one is that LLM is AI." The second is, "I would say, "No, Copilot is the new Clippy", I'm even playing with a paperclip right now."
Ben's newsletter subscription is similar to Gary's.
Now, Benioff is just another tech CEO, though of the second-largest software company in the world. My current favorite quote of the moment is François Chollet's in his presentation at AGI-24: "LLMs do not have zero intelligence, which is interesting."
I think your analysis is accurate. But, I'm seeing the transformation in my organisation. All comes down to choosing the branch of AI wisely, integrating well, and mitigating for AI's failings with systems engineering techniques. We were doing forms of 'deep' ANNs since the 70s before folks started calling it deep in the 90s and in some cases with what folks now call neuromorphic computing. A lot of what I see today is really old news, we just didn't tell anyone about it.
sure, but reread what she said, it was re powerful people and in business. the survey is reasonably representative of that. so your objection is not relevant in this context.
as i said, “reasonably representative’. you are nitpicking in a way that unproductive and misleading relative to her post and mine, strawmanning my claim. I therefore won’t continue the dialog with you.
"...waiting to see if it will work"?? Most people I know use GenAI or OTHER AI applications not GenAI and are pretty clear that "it works". Not sure what you are talking about here. First, AI is not just GenAI and second, GenAI works really well in many areas where the diminishing problem of "hallucinations" is less and less relevant....
One of the problems in talking about "AI" is that it's so ill defined. A lot of what we've been calling "machine learning" for the past decade or two is now called "AI". But, since the current AI boom came out of generative AI, it's not unreasonable to assume this is what she's referring to.
I don't think Gary is denying that generative AI has its applications. It's the misapplications and overpromising and ridiculous prognostication that he complains about. Sadly, these are plentiful.
I highly recommend going through the DORA report (from Google Cloud), page 18 has an interesting sentence saying „The vast majority of respondents (81%)
reported that their organizations have
shifted their priorities to increase their
incorporation of AI into their applications
and services. 49.2% of respondents even
described the magnitude of this shift as
being either “moderate” or “significant.”
=> so from that DORA report it’s the opposite.
I‘m sure there are still a lot of folks, executives not into it, but that’s because they might have a different business and other issues right now, eg Automotive companies 😅
Not sure I agree. CNBC fan list are techies or knowledgeable traders who know how to find a new app and not afraid to try it. All my friends (at least 95% of them) in a fairly peaceful New England coastal town, but with a good education and affluent enough to all have a laptop, have never heard of Copilot. They’ve heard of AI and ChatGPT but not consciously tried either! I think this influencer’s got it right.
sure, but reread what she said, it was re powerful people and in business. the survey is reasonably representative of that. so your objection is not relevant in this context.
1) everyone I know in the real working world, as opposed to the world of AI pundits, is using Co-pilot or some version. EVERYONE realizes that this is a new technology, and uses it carefully, while learning about it strengths and limitations, just like another other technology tool. 2) EVERYONE I know is gaining major benefits from just the use of really good chatbots, that will only get better. Many of the users benefitting greatly, that Gary does not talk about much is coders. Every coder I know is basically using Co-pilot/ChatGPT, etc. to improve coding, generate unit tests, save lots of time, etc. Here, no one is waiting around to see if GenAI "works." I just used ChatGPT-4o to walk me through installing a complex development environment for LLMs on my PC, and yes, it worked really well.....
The suffocating hype is a major source of my skepticism. Stop telling me how great it is, and start giving me concrete and measurable examples. Not examples that fall apart with the tiniest bit of scrutiny. The Information had a post a few days ago about using it in sales and instructing it to insert typos so that customers wouldn’t realize it’s AI generated. Tricking people is not how we build trust, and trust is the foundation of so many good things.
Open up the data, algorithms and code for independent outside testing and benchmarking.
Otherwise it’s not science or even engineering but just voodoo.
That’s actually what OpenAI promised at the beginning, at any rate.
That about the suffocating naysayers, doomers, nattering nabobs saying nothing is working, AI sucks, etc.? Interesting that these people want to have it both ways. AI is not ready for primetime! But we really really need to worry about the existential threat from AI....
A lot of the doomers are trying to scare us into thinking it’s better than it is. Almost worse, I agree!
The naysayers and the doomers are definitely not in the same camp. Speaking as a proud naysayer, I think AI doomers are the most outrageous hypesters of them all. Sam Altman's got nothing on Eliezer Yudkowsky when it comes to irresponsible, pseudo-religious prognistication.
I see the advertisements that read along the lines of, "Use AI to provide a personal response." I shake my head that they've not only twisted the definition of "intelligence", but they're twisting the definition of "personal". Yes, anything that is truly "AI" should not be applied as a trick.
I'm also seeing more shrill "AI-good-cause-washing", where genAI is being pushed as the solution to society's various ills. Eric Schmidt's "AI will solve climate crisis" is perhaps the most absurd and high profile example, but I attended a healthcare conference last week where nVidia were pushing their foundation model insta-solutions to the various challenges in the drug discovery industry. This was met by much weary eye rolling from executives for whom this isn't their first such rodeo, but perhaps plays better in the general media. I take all this as signs that things are slowing down worryingly for the world's largest tech companies.
The adjective Nvidias comes to mind
Clever
I’m surprised too that Cassie’s post exists and that it is still up.
Thanks for bringing this up Gary.
Same! Don't know her personally but it seemed way out of left field. I wondered whether she had really written it or whether she's got a team posting on her behalf. In which case... the end result is the same.
Sounds like something ChatGPT would write
FabricateGPT
Yes perhaps it did...
It's marketing. She's creating fear and FOMO to market her consulting services. Even smart people like her can do crappy marketing.
I also think that Kazyrkov got it backwards. I would say this:
*The most powerful people in the world have been bamboozled by AI hype.
*They over-estimate AI's capabilities (at least the current state of the art).
*They over-estimate the value to their business and under-estimate the work required to receive value.
*They think that they understand it, but do not. The current crop of models are LANGUAGE models, not cognitive models, not reasoning models.
*They think that they know how to use it. Co-pilots can be useful, but they are only a tiny fraction of what the current AI models can do.
*They think that an AI product is one that simply repackages chatbots.
*They have no idea of the critical role played by data in all of these AI models. They under-estimated the cost of getting good data and over-estimate the accuracy of the system.
*Some of these powerful people fear what they guess AI will do and they seek to regulate it. But, being fooled by the hype, they risk focusing the regulations on the wrong problems, and missing the critical targets.
I think the below explains one of the reasons:
OpenAI models fail own benchmark test?
“OpenAI has released a new benchmark test to assess the factual accuracy of AI models and found their own lacking. It ran its own AI models through the test and found that its best-performing model–GPT-o1—scored just 42.7%, with its next best model—GPT-4-o—scoring a measly 38.2%. Its smallest model—GPT-4o mini—only scored 8%, but OpenAI has blamed this on its small size, saying it has ‘little knowledge about the world'. Although the GPT scores are poor, Anthropic's best model, Claude-3.5 Sonnet, only got 28.9% of questions right, underscoring the worrying factual accuracy of AI models, and perhaps highlighting that they shouldn't be used as a single source of information.”
Also, based on conducting a pilot in my organization, I have found that it is impossible for most people to justify the $30/month cost. The productivity gains are very limited with Microsoft co-pilot. In some cases, people spent more time verifying the information produced by the copilots rather than performing the activity without one. In other cases, some people blindly trusted the information.
Integrating copilot or GenAI into enterprise applications is risky unless you can trust the information produced by these products. We are not there yet, and we may not be for a while or ever with current models.
What the heck is a "decision scientist?" But yeah, plenty of people including me have already "tried out some AI tools." https://davidhsing.substack.com/p/neural-network-fail-compilation
If I had to guess, I’d say the Chief Decision Scientist “ is someone who makes the final decisions for all the indecisive people at Google.
The latter would presumably be “indecision scientists” (if they are scientists) and “indecision engineers” if engineers.
It is probably no coincidence that 'influencer' and 'influenza' are such similar words. The things they connotate are both to be avoided if you value your health.
While he's a Microsoft competitor, Ben Thompson's interview with Marc Benioff outlined the limits and applications of LLMs. As Salesforce's CEO, Benioff is actively integrating LLM technology into Salesforce's products.
With that, a couple of MB punchlines. My favorite is, "there's two places customers have been dramatically misled, one is that LLM is AI." The second is, "I would say, "No, Copilot is the new Clippy", I'm even playing with a paperclip right now."
Ben's newsletter subscription is similar to Gary's.
Now, Benioff is just another tech CEO, though of the second-largest software company in the world. My current favorite quote of the moment is François Chollet's in his presentation at AGI-24: "LLMs do not have zero intelligence, which is interesting."
When I saw the title of this post, I was thinking it was about influencers that *are* AIs, not influencers *talking about* AI.
And now I wonder how much overlap there is.
She could have a bright future at a company like Altria.
ouch
Don't hold back Gary. I'd just say it how it is.
I think your analysis is accurate. But, I'm seeing the transformation in my organisation. All comes down to choosing the branch of AI wisely, integrating well, and mitigating for AI's failings with systems engineering techniques. We were doing forms of 'deep' ANNs since the 70s before folks started calling it deep in the 90s and in some cases with what folks now call neuromorphic computing. A lot of what I see today is really old news, we just didn't tell anyone about it.
79% of survey respondents != 79% of all humans
sure, but reread what she said, it was re powerful people and in business. the survey is reasonably representative of that. so your objection is not relevant in this context.
Only if 79% of respondents accurately reflects 79% of all “powerful people” (for some definition of powerful).
as i said, “reasonably representative’. you are nitpicking in a way that unproductive and misleading relative to her post and mine, strawmanning my claim. I therefore won’t continue the dialog with you.
I would be utterly shocked if 10% of the powerful people at my work had used Copilot. They’re stuck in meetings 10 hours a day.
"...waiting to see if it will work"?? Most people I know use GenAI or OTHER AI applications not GenAI and are pretty clear that "it works". Not sure what you are talking about here. First, AI is not just GenAI and second, GenAI works really well in many areas where the diminishing problem of "hallucinations" is less and less relevant....
One of the problems in talking about "AI" is that it's so ill defined. A lot of what we've been calling "machine learning" for the past decade or two is now called "AI". But, since the current AI boom came out of generative AI, it's not unreasonable to assume this is what she's referring to.
I don't think Gary is denying that generative AI has its applications. It's the misapplications and overpromising and ridiculous prognostication that he complains about. Sadly, these are plentiful.
I highly recommend going through the DORA report (from Google Cloud), page 18 has an interesting sentence saying „The vast majority of respondents (81%)
reported that their organizations have
shifted their priorities to increase their
incorporation of AI into their applications
and services. 49.2% of respondents even
described the magnitude of this shift as
being either “moderate” or “significant.”
=> so from that DORA report it’s the opposite.
I‘m sure there are still a lot of folks, executives not into it, but that’s because they might have a different business and other issues right now, eg Automotive companies 😅
Not sure I agree. CNBC fan list are techies or knowledgeable traders who know how to find a new app and not afraid to try it. All my friends (at least 95% of them) in a fairly peaceful New England coastal town, but with a good education and affluent enough to all have a laptop, have never heard of Copilot. They’ve heard of AI and ChatGPT but not consciously tried either! I think this influencer’s got it right.
sure, but reread what she said, it was re powerful people and in business. the survey is reasonably representative of that. so your objection is not relevant in this context.
Again???
give us a solution
The solution exists between your ears.
OK…thank you.
As copilot is more aligned to business use than the general public, I’m not entirely surprised that some have not heard of it. Maybe🤷🏾♂️
1) everyone I know in the real working world, as opposed to the world of AI pundits, is using Co-pilot or some version. EVERYONE realizes that this is a new technology, and uses it carefully, while learning about it strengths and limitations, just like another other technology tool. 2) EVERYONE I know is gaining major benefits from just the use of really good chatbots, that will only get better. Many of the users benefitting greatly, that Gary does not talk about much is coders. Every coder I know is basically using Co-pilot/ChatGPT, etc. to improve coding, generate unit tests, save lots of time, etc. Here, no one is waiting around to see if GenAI "works." I just used ChatGPT-4o to walk me through installing a complex development environment for LLMs on my PC, and yes, it worked really well.....
“real working world, as opposed to the world of AI pundits” harsh!! 😄
It's kind of fun to play with. But I find all of them too repetitive.
Linkdin is the new facebook really. Its full of grifters, thirst traps, influencers and bots.