Essentially every conversation about “driverless cars” over the last decade has to be rethought — with important implications as well for “AGI timelines”.
I agree with every point you made above. The whole thing is indeed a farce. Also, all the talk about progress being made toward solving AGI that we hear coming from generative AI experts and AI executives is pure snake oil.
Deep learning (generative AI) will never get us closer to AGI regardless of their undeniable usefulness in some applications. It is a giant step backward for humanity's quest for AGI in my opinion. It sent everyone chasing after a red herring. It sucked all the funding out of research efforts on alternative AI models.
AI research should be focused primarily on systematic generalization. Without fully generalized perception, the corner case problem that plagues both self-driving systems and LLMs cannot be solved. Deep learning is based on objective function optimization which is the opposite of generalization. It is useless for AGI.
The deep learning era will soon come to an end. It's time to look under a different lamppost.
I think it's best to actually reverse things: deep learning has made actually useful products like MidJourney. AI Art is incredibly useful and valuable, and we've found some other uses of this technology as well, like coding assistance.
AGI, meanwhile, is a weird religious belief that has no basis in reality.
I don't think deep learning is going to come to an end. Deep learning is useless for making intelligence, but really, making a true synthetic intelligence isn't even necessarily particularly useful. Deep learning is useful for producing certain types of things.
Hallucinations are a problem with something where you care about it being factual or consistent, but if you don't need to care about that - like with art images - there's a huge amount of value there.
Deep learning can do things like image correction, which is actually useful technology. There's a lot of SFX work where it is potentially very useful. Same with generating content for things like video games and other virtual environments.
I love deep learning and I agree that it is a fascinating and useful technology. I've argued in the past that DL will continue to be highly useful even after AGI is solved. It's just not useful to solving AGI.
DL experts, especially influencers like Yann LeCun, should refrain from insisting that DL will be the foundation of AGI. This is injurious to AGI research in my opinion.
It is true that current approaches cannot robustly generalize from a small set of diverse and rare observations.
The important question here is if humans are really that clever, or we just draw from our very vast amount of well-integrated and well-modeled knowledge about the world that we acquire from our experience.
If we are really that clever, it means that we need to find a very smart model. That task looks daunting to me, likely exceeding our current level of knowledge and technology.
If we just need to thoroughly and methodically map the world, that looks like a vast problem that will keep us busy for a decade or two, but a lot more tractable given our current resources.
In deep learning, fitting a curve to a given dataset will only "generalize" to that distribution. I would not call curve fitting generalization but DL experts do for some reason. Interpolation is not generalization. True generalization is the ability to instantly perceive any pattern or object without recognizing it. We would not survive without this ability. DL cannot generalize without recognition. That's a fatal flaw in my opinion.
I believe that AGI can conceivably be solved much sooner than most experts believe. The most important move we can make in that direction is to immediately abandon deep learning and everything associated with it. I believe that everything in cognition depends on systematic generalization. We need a breakthrough in this area but it can only happen if many researchers are focusing on it.
You say: "True generalization is the ability to instantly perceive any pattern or object without recognizing it.".
I will respectfully say that your focus on "generalization" alone and belief that one single successful strategy will crack AI is very short-sighted and doomed to failure.
People do generalize better than machines, but that's because we have better representations of the world and deeper knowledge.
People do very badly in unfamiliar circumstances and with little practice.
AGI is not a problem to be cracked, it is infrastructure to be built.
According to you, all the world's researchers have been playing dumb for 20 years.
You have no understanding of the limits of generalization and of the large amount of phenomena that need modeling and work that need to be done to get to AGI.
Generalization is only one piece of the puzzle, and even then, it has to be robust generalization based on data.
MidJourney only recombines images via a process called stable diffusion. Human intelligence invented that process. There's nothing creative about stable diffusion. It's certainly not generalization. MidJourney has no understanding of beauty.
1) Diffusion processes like Midjourney do not recombine images. This is obvious if you look at how these programs function - the total size is very small relative to the training set, far too small to store the images that it was trained on, and the images are not collages.
2) MidJourney isn't a Stable Diffusion fork; while it uses a diffusion process as part of its algorithm, it is not actually just a diffusion program.
They way they actually work is by mathematically deriving the statistical properties of images with certain words or phrases in their prompts and then applying those statistical properties onto a randomized field in order to create de novo images that have never existed previously. It will create images of things that have never existed previously, and in fact, virtually always does so outside of deliberately trying to recover very famous, commonly reproduced images (like the Mona Lisa).
The end program has no source images to interpolate from, it's just a bunch of math. Which is why you can create things that have never existed before, like MacFarlane Toys figurines of characters who never had such things made, or creating an OC Macfarlane Toyas figurine. That's just one obvious example, where you create something in a style but the thing you're creating has never existed.
It's not intelligent but it creates novel coherent images, which is, in effect, "creativity".
While it doesn't "understand" beauty, the mathematical processes it encodes do in fact code for beauty, which is why the images it produces look pretty good overall, and you end up with attractive people showing up in your images. In fact, the very fact that it works suggests that there's some sort of empirical component to beauty.
Remote assisted driving could be a nightmare. I have recently had solar panels fitted and the user statistics is handled remotely when the internet and the remote computer is not overloaded. Imagine driving at speed along a motorway and all communications are lost (might even be a major solar flare disrupting satellite cover). ANy safe system must be self-contained.
Nov 22, 2023·edited Nov 22, 2023Liked by Gary Marcus
Many of the people posting in the comment section for this story sound like you have programming experience in the field of engineering automated automobiles.
How many of you have ever worked as a driving professional in an occupation like cabdriver (mostly on surface streets) for at least one year and >40,000 miles, or as an over the road truck driver for at least one calendar year and >80,000 miles? In all sorts of traffic and road conditions, lane closures for maintenance work on roads and bridges, hazardous weather, at all hours, urban, suburban, rural, and freeway routes, with no traffic infractions or accidents?
( The "professional" part is important, because it implies driving for many hours day after day, whether you feel like it or not; and running someone else's miles, not on your own preference or schedule, sometimes navigating routes that you've never driven, to places you've never been, under driving conditions that you'd rather not have to contend with. And, if you stay out there long enough, eventually encountering unpredictable circumstances and unusual, unforeseen challenges. Also, nobody lasts in the business unless they're safe drivers with good driving records. Insurance, you know.)
If you're members of a team, how many of the team members have that amount of hands-on competence, doing the work professionally?
Failing that, how many have professional driving experience that approaches the amount specified in my first question?
I really would like to get at least one reply to those questions, whether affirmative or negative.
Gary, remote assist is nothing new and that is well known within the AV sector. Everyone uses it - and in the initial ‘learning’ phases, do so quite intensively. Contrary to the leading premise of this commentary, remote assist is neither a rumour, nor is it a surprise. Waymo, for example, spoke about it openly many years ago. Presumably Waymo (which has been much safer than GM) relies on remote assist much less now - they may have published info about that, I don’t know.
In the distant or impossible future when full Level 5 becomes commonplace and trustworthy, humans will still be in the loop to deal with special situations, emergencies, etc.
And, AVs don’t need to be perfect. They just need to significantly improve on the 42,000 traffic deaths per year in the US (2021 statistic).
I keep hearing this pitch about safety and improving on human-caused accidents and deaths. My questions are: what data will we have to prove this? At what costs? Against what other alternatives (like less driving and public transport) which would cost less? What compromises will need to made by the consumer to achieve this (privacy, insurance cost, regulations)? So this is not "just only" proposition. It deserves much more analysis before we believe that statement.
The data indicates that some 90% of vehicle crashes are due to human factors such as fatigue, intoxication, distraction etc. It’s unlikely that AVs will achieve public acceptance if they are anywhere close to as accident prone as humans. Re cost, I agree that less driving and public transit (which could also be automated) are better and more cost effective than single person AVs. As for the other compromises, privacy might be worse than in today’s highly computerized cars - along with location tracking smartphones - but not by much. Privacy should be addressed now for the current reality. Insurance should be cheaper with a decline in accidents & car theft. Not sure what you mean by regulations.
Thank for the thoughtful response. I respect what you have to say. I think the larger point I am making is that self-driving is not at the peak of customer demand for vehicle innovation. Fuel efficiency and range may be rated higher, I don't think seat belts increase customer demand for cars against all the other factors driving demand. So even if they are to some degree safer it doesn't mean people will use them or care very much. Demand is about too many other factors. Driver's education, improved road conditions and more traffic enforcement might also lower accidents. On the regulation front, nobody knows but with most new technologies eventually it comes around A ticket for having your seat belt on in CA could cost you hundreds of dollars. A child not in a car seat even more. We can only imagine. Texting while driving is another. If you are being monitored and recorded by the system what regulations will follow that?
The problem of self-driving cars has three sides, technical, economic, and social acceptance.
1. Technical: It's best to dissociate discussion of self-driving cars from "AGI". AGI is a red herring. Self-driving cars are a specialized form of AI that relies heavily on ML and huge amounts of data, but is nothing like LLMs. Self-driving car AI lacks general reasoning and world knowledge and all that, but that doesn't prevent them from being very effective in the majority of relatively routine driving conditions, which is actually quite big and diverse. They are probably safer than the average human driver in average conditions. Lacking a human mind, they perform differently---and often worse---in the long tail of unusual situations. No surprise there, there are different types of intelligence, with different strengths and weaknesses. The shape and tractability of the tail can be known only from real-world deployments.
2. The long, fat, tail of non-routine situations is being addressed with human remote monitoring and assistance. Hooray! Autonomous vehicle companies are smart to over-staff this function for three reasons: (a) collect data to push the tail back; (b) be conservative about safety and disruption; (c) meet peak demand loads. Whether this is economical or not in the long run depends on the learning rate; driving conditions in which the vehicles are deployed; and safety/disruption/cost tradeoffs. We are in violent agreement (along with industry analyst Brad Templeton) not only that the public deserves transparency here, but that it is in the companies' interest to provide it. It is way-premature to make a call that these things are not economically viable. The savings to society from getting bum human drivers off the road are monumental. The gamble that investors are making is that this saving can be harvested. That's in addition to the benefits from increased options for mobility.
3. While the AI-technology and economic calculations are churning away, the biggest challenge now is social acceptance. LLM alarm and skepticism have helped to push public sentiment against AI overall. Deceptive over-promotion of "full self-driving" hurts as well. Every mishap will make the news in a way that human-caused car tragedies do not. This is just how the collective mind works, so management of expectations is paramount. In our society, fortunately, the best strategy is transparency. Then, at least the debates could be based on well-established, open facts instead of us having to finely parse spin found in New York Times articles.
"No fantasy about AGI thus far has survived contact with the real world." ain't that a fact.
Maybe it should read "No conviction about around-the-corner AGI thus far has survived contact with the real world." Human convictions are funny things, and are quite resistant to reasoning and observations. What — hopefully — the crash of AGI-expectations is going to bring us is some proper attention to the 'fantasies' we constantly have about them. AI might teach us above all a useful lesson about 'human intelligence'.
I think the hard lessons learned so far about human intelligence and AI are that (a) nothing is fast easy (b) there is not grand plan to our mind (c) we will have to develop many models for specific circumstances (d) incremental progress on specific issues followed by generalization when a pattern is seen beats the alternatives.
Your article shows that people are opinionated and stubborn, and are quite likely to get it wrong when making big decisions. That's why in the AI field we should focus on incremental work and measurable improvements, rather than grand theories and bold timelines.
I very much agree that self driving cars continues to feel further off than people imagined. LLMs have tons of training data, but cars don’t have much, and taking similar strategies will struggle there. So I guess it makes sense they’re collecting a ton of data (from local or remote driver assist). TBD whether that gets us anywhere...
But that said, I’m not sure the goal of self driving was ever “AGI capable of doing any task”, are you refuting someone who said that self driving cars would give us AGI?
It also sounds like you’re saying transformers are the cause of hallucinations. I had always assumed it was the “next token prediction” of positive-only dataset examples in pretraining that gives it the confidence in its hallucinations. Can you share more about why you think the transformer is fundamentally at fault and will lead to hallucinations if used in self driving AI? Would be curious how you make that connection
Seems more and more likely that AI is well down the transformer off-ramp, about to get seriously lost, unable to find its way back to the main road. Or to strain yet another metaphor, AI winter is coming and it looks like a long one.
If the consumer base tires of unreliable-to-deadly, over-hyped "AI solutions," funding into AI research (especially private sector funding) is likely to dry up.
My best hope for AGI in my lifetime is a dramatically increased one (which will more likely come from highly targeted ML models than say, an LLM magically curing cancer at the behest of a well-crafted text prompt.)
Actually, Cruise has said that the remote-assisted-self-driving requires roughly 2 minutes of remote-assistance per 1 hour driving. The rest is cleaners, and other staff. You might need about 1 human remote assistant per 15 cars or so.
It is an interesting metric. But th eL5 is AGI-like and that is really not around the corner (except in the convictions of many since 2005). Gates was all-in on AI in the 80's and 90's. It all failed. Musk has been all-in on autonomous cars. I think Gary is going to be right. At some moment, the convictions no longer survive.
This still says nothing about Waymo though. It is good to have more specifics. I posted something in another comment further down. And mentioning Musk doesn't really help. Musk is not a serious person when it comes to self-driving cars.
I expect it really doesn't matter what Waymo is specifically doing unless they also use radically different computing hardware.
I agree with Gary: the 'assisted driving' (either remote or locally) is the best current (digital) AI-techniques can handle. The problem is fundamental, not linked to a specific implementation.
Gary, thanks for the link. Yes, we need more info. There is also broad agreement that the systems need to get more mature. It is not clear however if what we encounter are fundamental issues, or there is more engineering problems to be solved.
Intelligence has three scalable dimensions: knowledge (information), inventiveness (problem-solving ability, what I suspect you are referring to as "algorithms"), and compute (hardware). It's not "just about algorithms", and neither is it "just about hardware". For AGI, you need (lots and lots of) all three.
I have my reasons to suspect it will in the end be about the hardware. I think digital hardware is fundamentally limited in terms of its expressive power. You might compare it with "between every two integers there are infinitely many reals". Trying to approximate reals with integers is going to fail when there is "contact with the REAL world". We can mathematically approximate some signals well enough with integers, but definitely not all. If I am right, it doesn't really matter what Waymo does — algorithm wise, as long as they work in a purely digital fashion. But I may be wrong, of course.
The human eyes, ears, and neurons have only very limited sensitivity to signals. And synapses coarsely approximate any signals going through them, as they use chemical and electrical gradients.
Computer hardware is much more precise by many orders of magnitude. A floating point number has 16 digits of precision. Distance to the nearest star is 3.8 10^16 m.
The difficulty in creating AI is not there. It is at the high level. The human brain is just very large and stores a huge amount of information and models of the world.
Could you elaborate on how AGI is a requirement for self-driving?
By the way, I think the bulk of driving doesn't need L5, with L4 it would be largely enough...
About Waymo, I have an anecdote. Some years ago I went to a Faculty Summit at Google's HQ representing my university. As a part of the program, Larry Page talked with us for an hour or so, with no specific points to discuss; "ask me whatever you want," he said.
During the conversation, Larry recounted how he took a PhD topic with this advisor, Terry Winograd. It went as follows: Terry proposed Larry to take one of two projects:
- One of them was to develop self-driving cars;
- The other was to investigate the structure of the internet with the goal of improving search.
After agonizingly pondering the two projects for some days, he decided to take the second one, with the results that we all know.
But in the back of Larry's head the self-driving idea somehow stayed.
Years after Google made many millions in profit and started to diversify, the self-driving project came back to life under Waymo.
One interesting bit about the approach Larry wanted to take in self-driving is that he preferred to skip assisted driving altogether. That's why some early Waymo prototypes didn't have steering wheel at all.
There you have the anecdote. Of course, it doesn't tell much about how self-driving will end. We all can agree that the difficulty of self-driving in real life was HUGELY underestimated.
This week I'll publish my take on where self-driving is heading as a Medium post.
This is my hypothesis: Prob(Developing quantum computers that can do something useful in our lifetime) ≥ Prob(String theory will ever be experimentally verified) = Prob(Developing level 5 autonomous cars) = 0 > Prob(Developing a AGI system) >> Prob(Developing a ASI system).
"Remotely-assisted" driving? What...?? You mean someone +100 miles away is helping you to steer your car for you, like in a video game? And we're supposed to accept this new paradigm as a substitute for the advertised hype that is L5? This has to be a joke. Or at least a smoke and mirrors magic trick at the publics expense. Forget frustration, my emotions are turning to anger at the seemingly sheer dishonest propagandist skullduggery of it all...
I agree with every point you made above. The whole thing is indeed a farce. Also, all the talk about progress being made toward solving AGI that we hear coming from generative AI experts and AI executives is pure snake oil.
Deep learning (generative AI) will never get us closer to AGI regardless of their undeniable usefulness in some applications. It is a giant step backward for humanity's quest for AGI in my opinion. It sent everyone chasing after a red herring. It sucked all the funding out of research efforts on alternative AI models.
AI research should be focused primarily on systematic generalization. Without fully generalized perception, the corner case problem that plagues both self-driving systems and LLMs cannot be solved. Deep learning is based on objective function optimization which is the opposite of generalization. It is useless for AGI.
The deep learning era will soon come to an end. It's time to look under a different lamppost.
I think it's best to actually reverse things: deep learning has made actually useful products like MidJourney. AI Art is incredibly useful and valuable, and we've found some other uses of this technology as well, like coding assistance.
AGI, meanwhile, is a weird religious belief that has no basis in reality.
I don't think deep learning is going to come to an end. Deep learning is useless for making intelligence, but really, making a true synthetic intelligence isn't even necessarily particularly useful. Deep learning is useful for producing certain types of things.
Hallucinations are a problem with something where you care about it being factual or consistent, but if you don't need to care about that - like with art images - there's a huge amount of value there.
Deep learning can do things like image correction, which is actually useful technology. There's a lot of SFX work where it is potentially very useful. Same with generating content for things like video games and other virtual environments.
I love deep learning and I agree that it is a fascinating and useful technology. I've argued in the past that DL will continue to be highly useful even after AGI is solved. It's just not useful to solving AGI.
DL experts, especially influencers like Yann LeCun, should refrain from insisting that DL will be the foundation of AGI. This is injurious to AGI research in my opinion.
It is true that current approaches cannot robustly generalize from a small set of diverse and rare observations.
The important question here is if humans are really that clever, or we just draw from our very vast amount of well-integrated and well-modeled knowledge about the world that we acquire from our experience.
If we are really that clever, it means that we need to find a very smart model. That task looks daunting to me, likely exceeding our current level of knowledge and technology.
If we just need to thoroughly and methodically map the world, that looks like a vast problem that will keep us busy for a decade or two, but a lot more tractable given our current resources.
In deep learning, fitting a curve to a given dataset will only "generalize" to that distribution. I would not call curve fitting generalization but DL experts do for some reason. Interpolation is not generalization. True generalization is the ability to instantly perceive any pattern or object without recognizing it. We would not survive without this ability. DL cannot generalize without recognition. That's a fatal flaw in my opinion.
I believe that AGI can conceivably be solved much sooner than most experts believe. The most important move we can make in that direction is to immediately abandon deep learning and everything associated with it. I believe that everything in cognition depends on systematic generalization. We need a breakthrough in this area but it can only happen if many researchers are focusing on it.
You say: "True generalization is the ability to instantly perceive any pattern or object without recognizing it.".
I will respectfully say that your focus on "generalization" alone and belief that one single successful strategy will crack AI is very short-sighted and doomed to failure.
People do generalize better than machines, but that's because we have better representations of the world and deeper knowledge.
People do very badly in unfamiliar circumstances and with little practice.
AGI is not a problem to be cracked, it is infrastructure to be built.
You have no understanding of generalization. This exchange is not fruitful. Thanks.
According to you, all the world's researchers have been playing dumb for 20 years.
You have no understanding of the limits of generalization and of the large amount of phenomena that need modeling and work that need to be done to get to AGI.
Generalization is only one piece of the puzzle, and even then, it has to be robust generalization based on data.
MidJourney can create images that have never existed before. It can, in fact, generalize.
The thing is, it's not intelligent at all.
The idea that something has to be intelligent to be useful is wrong.
MidJourney only recombines images via a process called stable diffusion. Human intelligence invented that process. There's nothing creative about stable diffusion. It's certainly not generalization. MidJourney has no understanding of beauty.
No. This is completely wrong.
1) Diffusion processes like Midjourney do not recombine images. This is obvious if you look at how these programs function - the total size is very small relative to the training set, far too small to store the images that it was trained on, and the images are not collages.
2) MidJourney isn't a Stable Diffusion fork; while it uses a diffusion process as part of its algorithm, it is not actually just a diffusion program.
They way they actually work is by mathematically deriving the statistical properties of images with certain words or phrases in their prompts and then applying those statistical properties onto a randomized field in order to create de novo images that have never existed previously. It will create images of things that have never existed previously, and in fact, virtually always does so outside of deliberately trying to recover very famous, commonly reproduced images (like the Mona Lisa).
The end program has no source images to interpolate from, it's just a bunch of math. Which is why you can create things that have never existed before, like MacFarlane Toys figurines of characters who never had such things made, or creating an OC Macfarlane Toyas figurine. That's just one obvious example, where you create something in a style but the thing you're creating has never existed.
It's not intelligent but it creates novel coherent images, which is, in effect, "creativity".
While it doesn't "understand" beauty, the mathematical processes it encodes do in fact code for beauty, which is why the images it produces look pretty good overall, and you end up with attractive people showing up in your images. In fact, the very fact that it works suggests that there's some sort of empirical component to beauty.
Remote assisted driving could be a nightmare. I have recently had solar panels fitted and the user statistics is handled remotely when the internet and the remote computer is not overloaded. Imagine driving at speed along a motorway and all communications are lost (might even be a major solar flare disrupting satellite cover). ANy safe system must be self-contained.
I thought there had been only 2 AI Winters. My mistake.
Wikipedia lists quite a few more. https://en.wikipedia.org/wiki/AI_winter
Anyone feel a chill?
Winter is coming.
Many of the people posting in the comment section for this story sound like you have programming experience in the field of engineering automated automobiles.
How many of you have ever worked as a driving professional in an occupation like cabdriver (mostly on surface streets) for at least one year and >40,000 miles, or as an over the road truck driver for at least one calendar year and >80,000 miles? In all sorts of traffic and road conditions, lane closures for maintenance work on roads and bridges, hazardous weather, at all hours, urban, suburban, rural, and freeway routes, with no traffic infractions or accidents?
( The "professional" part is important, because it implies driving for many hours day after day, whether you feel like it or not; and running someone else's miles, not on your own preference or schedule, sometimes navigating routes that you've never driven, to places you've never been, under driving conditions that you'd rather not have to contend with. And, if you stay out there long enough, eventually encountering unpredictable circumstances and unusual, unforeseen challenges. Also, nobody lasts in the business unless they're safe drivers with good driving records. Insurance, you know.)
If you're members of a team, how many of the team members have that amount of hands-on competence, doing the work professionally?
Failing that, how many have professional driving experience that approaches the amount specified in my first question?
I really would like to get at least one reply to those questions, whether affirmative or negative.
Gary, remote assist is nothing new and that is well known within the AV sector. Everyone uses it - and in the initial ‘learning’ phases, do so quite intensively. Contrary to the leading premise of this commentary, remote assist is neither a rumour, nor is it a surprise. Waymo, for example, spoke about it openly many years ago. Presumably Waymo (which has been much safer than GM) relies on remote assist much less now - they may have published info about that, I don’t know.
In the distant or impossible future when full Level 5 becomes commonplace and trustworthy, humans will still be in the loop to deal with special situations, emergencies, etc.
And, AVs don’t need to be perfect. They just need to significantly improve on the 42,000 traffic deaths per year in the US (2021 statistic).
I keep hearing this pitch about safety and improving on human-caused accidents and deaths. My questions are: what data will we have to prove this? At what costs? Against what other alternatives (like less driving and public transport) which would cost less? What compromises will need to made by the consumer to achieve this (privacy, insurance cost, regulations)? So this is not "just only" proposition. It deserves much more analysis before we believe that statement.
The data indicates that some 90% of vehicle crashes are due to human factors such as fatigue, intoxication, distraction etc. It’s unlikely that AVs will achieve public acceptance if they are anywhere close to as accident prone as humans. Re cost, I agree that less driving and public transit (which could also be automated) are better and more cost effective than single person AVs. As for the other compromises, privacy might be worse than in today’s highly computerized cars - along with location tracking smartphones - but not by much. Privacy should be addressed now for the current reality. Insurance should be cheaper with a decline in accidents & car theft. Not sure what you mean by regulations.
Thank for the thoughtful response. I respect what you have to say. I think the larger point I am making is that self-driving is not at the peak of customer demand for vehicle innovation. Fuel efficiency and range may be rated higher, I don't think seat belts increase customer demand for cars against all the other factors driving demand. So even if they are to some degree safer it doesn't mean people will use them or care very much. Demand is about too many other factors. Driver's education, improved road conditions and more traffic enforcement might also lower accidents. On the regulation front, nobody knows but with most new technologies eventually it comes around A ticket for having your seat belt on in CA could cost you hundreds of dollars. A child not in a car seat even more. We can only imagine. Texting while driving is another. If you are being monitored and recorded by the system what regulations will follow that?
Maybe it's a good occasion to recall that Toyota's approach is precisely a driver assistance system, which was pesented in 2019 https://spectrum.ieee.org/ces-toyota-lifts-veil-from-driver-assist-system.
Essentially, the car avoids the driver from doing incorrect movement or making wrong decisions.
Time will tell if this 'modest' technological approach is the correct one for specific urban frameworks.
The problem of self-driving cars has three sides, technical, economic, and social acceptance.
1. Technical: It's best to dissociate discussion of self-driving cars from "AGI". AGI is a red herring. Self-driving cars are a specialized form of AI that relies heavily on ML and huge amounts of data, but is nothing like LLMs. Self-driving car AI lacks general reasoning and world knowledge and all that, but that doesn't prevent them from being very effective in the majority of relatively routine driving conditions, which is actually quite big and diverse. They are probably safer than the average human driver in average conditions. Lacking a human mind, they perform differently---and often worse---in the long tail of unusual situations. No surprise there, there are different types of intelligence, with different strengths and weaknesses. The shape and tractability of the tail can be known only from real-world deployments.
2. The long, fat, tail of non-routine situations is being addressed with human remote monitoring and assistance. Hooray! Autonomous vehicle companies are smart to over-staff this function for three reasons: (a) collect data to push the tail back; (b) be conservative about safety and disruption; (c) meet peak demand loads. Whether this is economical or not in the long run depends on the learning rate; driving conditions in which the vehicles are deployed; and safety/disruption/cost tradeoffs. We are in violent agreement (along with industry analyst Brad Templeton) not only that the public deserves transparency here, but that it is in the companies' interest to provide it. It is way-premature to make a call that these things are not economically viable. The savings to society from getting bum human drivers off the road are monumental. The gamble that investors are making is that this saving can be harvested. That's in addition to the benefits from increased options for mobility.
3. While the AI-technology and economic calculations are churning away, the biggest challenge now is social acceptance. LLM alarm and skepticism have helped to push public sentiment against AI overall. Deceptive over-promotion of "full self-driving" hurts as well. Every mishap will make the news in a way that human-caused car tragedies do not. This is just how the collective mind works, so management of expectations is paramount. In our society, fortunately, the best strategy is transparency. Then, at least the debates could be based on well-established, open facts instead of us having to finely parse spin found in New York Times articles.
"No fantasy about AGI thus far has survived contact with the real world." ain't that a fact.
Maybe it should read "No conviction about around-the-corner AGI thus far has survived contact with the real world." Human convictions are funny things, and are quite resistant to reasoning and observations. What — hopefully — the crash of AGI-expectations is going to bring us is some proper attention to the 'fantasies' we constantly have about them. AI might teach us above all a useful lesson about 'human intelligence'.
I think the hard lessons learned so far about human intelligence and AI are that (a) nothing is fast easy (b) there is not grand plan to our mind (c) we will have to develop many models for specific circumstances (d) incremental progress on specific issues followed by generalization when a pattern is seen beats the alternatives.
I was pointing at something else entirely, namely how our convictions ("AGI is around the corner!") work and why. See https://ea.rna.nl/2022/10/24/on-the-psychology-of-architecture-and-the-architecture-of-psychology/
Your article shows that people are opinionated and stubborn, and are quite likely to get it wrong when making big decisions. That's why in the AI field we should focus on incremental work and measurable improvements, rather than grand theories and bold timelines.
No, that is not what the article shows.
"No fantasy about AGI thus far has survived contact with the real world." - statement of the decade :)
I very much agree that self driving cars continues to feel further off than people imagined. LLMs have tons of training data, but cars don’t have much, and taking similar strategies will struggle there. So I guess it makes sense they’re collecting a ton of data (from local or remote driver assist). TBD whether that gets us anywhere...
But that said, I’m not sure the goal of self driving was ever “AGI capable of doing any task”, are you refuting someone who said that self driving cars would give us AGI?
It also sounds like you’re saying transformers are the cause of hallucinations. I had always assumed it was the “next token prediction” of positive-only dataset examples in pretraining that gives it the confidence in its hallucinations. Can you share more about why you think the transformer is fundamentally at fault and will lead to hallucinations if used in self driving AI? Would be curious how you make that connection
My p(doom) for AGI-in-my-lifetime just went up.
Seems more and more likely that AI is well down the transformer off-ramp, about to get seriously lost, unable to find its way back to the main road. Or to strain yet another metaphor, AI winter is coming and it looks like a long one.
If the consumer base tires of unreliable-to-deadly, over-hyped "AI solutions," funding into AI research (especially private sector funding) is likely to dry up.
My best hope for AGI in my lifetime is a dramatically increased one (which will more likely come from highly targeted ML models than say, an LLM magically curing cancer at the behest of a well-crafted text prompt.)
Waymo has much better product than Cruise (see https://www.understandingai.org/p/driverless-cars-may-already-be-safer and other posts of this author) The "rumors" you mentioned about frequently calling the call center may not be true for Waymo.
Actually, Cruise has said that the remote-assisted-self-driving requires roughly 2 minutes of remote-assistance per 1 hour driving. The rest is cleaners, and other staff. You might need about 1 human remote assistant per 15 cars or so.
It is an interesting metric. But th eL5 is AGI-like and that is really not around the corner (except in the convictions of many since 2005). Gates was all-in on AI in the 80's and 90's. It all failed. Musk has been all-in on autonomous cars. I think Gary is going to be right. At some moment, the convictions no longer survive.
This still says nothing about Waymo though. It is good to have more specifics. I posted something in another comment further down. And mentioning Musk doesn't really help. Musk is not a serious person when it comes to self-driving cars.
I expect it really doesn't matter what Waymo is specifically doing unless they also use radically different computing hardware.
I agree with Gary: the 'assisted driving' (either remote or locally) is the best current (digital) AI-techniques can handle. The problem is fundamental, not linked to a specific implementation.
We'll see.
https://x.com/aniccia/status/1721208695361847462?s=46
Gary, thanks for the link. Yes, we need more info. There is also broad agreement that the systems need to get more mature. It is not clear however if what we encounter are fundamental issues, or there is more engineering problems to be solved.
This is not about hardware, it is about algorithms.
Saying that it doesn't matter what the industry-leader is doing, and refusing any conversation about specifics is not a solid approach.
Intelligence has three scalable dimensions: knowledge (information), inventiveness (problem-solving ability, what I suspect you are referring to as "algorithms"), and compute (hardware). It's not "just about algorithms", and neither is it "just about hardware". For AGI, you need (lots and lots of) all three.
I have my reasons to suspect it will in the end be about the hardware. I think digital hardware is fundamentally limited in terms of its expressive power. You might compare it with "between every two integers there are infinitely many reals". Trying to approximate reals with integers is going to fail when there is "contact with the REAL world". We can mathematically approximate some signals well enough with integers, but definitely not all. If I am right, it doesn't really matter what Waymo does — algorithm wise, as long as they work in a purely digital fashion. But I may be wrong, of course.
The human eyes, ears, and neurons have only very limited sensitivity to signals. And synapses coarsely approximate any signals going through them, as they use chemical and electrical gradients.
Computer hardware is much more precise by many orders of magnitude. A floating point number has 16 digits of precision. Distance to the nearest star is 3.8 10^16 m.
The difficulty in creating AI is not there. It is at the high level. The human brain is just very large and stores a huge amount of information and models of the world.
Could you elaborate on how AGI is a requirement for self-driving?
By the way, I think the bulk of driving doesn't need L5, with L4 it would be largely enough...
About Waymo, I have an anecdote. Some years ago I went to a Faculty Summit at Google's HQ representing my university. As a part of the program, Larry Page talked with us for an hour or so, with no specific points to discuss; "ask me whatever you want," he said.
During the conversation, Larry recounted how he took a PhD topic with this advisor, Terry Winograd. It went as follows: Terry proposed Larry to take one of two projects:
- One of them was to develop self-driving cars;
- The other was to investigate the structure of the internet with the goal of improving search.
After agonizingly pondering the two projects for some days, he decided to take the second one, with the results that we all know.
But in the back of Larry's head the self-driving idea somehow stayed.
Years after Google made many millions in profit and started to diversify, the self-driving project came back to life under Waymo.
One interesting bit about the approach Larry wanted to take in self-driving is that he preferred to skip assisted driving altogether. That's why some early Waymo prototypes didn't have steering wheel at all.
There you have the anecdote. Of course, it doesn't tell much about how self-driving will end. We all can agree that the difficulty of self-driving in real life was HUGELY underestimated.
This week I'll publish my take on where self-driving is heading as a Medium post.
Gary, I haven’t seen you comment on the Waymo/Swiss Re study and what it says about Waymo’s system?
This is my hypothesis: Prob(Developing quantum computers that can do something useful in our lifetime) ≥ Prob(String theory will ever be experimentally verified) = Prob(Developing level 5 autonomous cars) = 0 > Prob(Developing a AGI system) >> Prob(Developing a ASI system).
"Remotely-assisted" driving? What...?? You mean someone +100 miles away is helping you to steer your car for you, like in a video game? And we're supposed to accept this new paradigm as a substitute for the advertised hype that is L5? This has to be a joke. Or at least a smoke and mirrors magic trick at the publics expense. Forget frustration, my emotions are turning to anger at the seemingly sheer dishonest propagandist skullduggery of it all...