'Snake oil' keeps ringing in my mind. Those who keep insisting that LLMs are a step closer to AGI are selling the same snake oil, in my opinion.
Generative AI, regardless of its utility, is a step backward in the search for AGI. It has almost nothing in common with intelligence as we observe it in humans and animals. AGI will not be found under this lamppost. We need new models, new theories.
I wonder if Gary's too young to remember Spinvox, 20 years ago in the early mobile phone boom, who claimed to have technology that could transcribe voice recordings (eg voicemail messages) to text,
and burned through $100m of investor cash
and, it turned out, had never made it past the "secret call-centre-style companies of human transcribers" stage...
Gary, I kind of get the sense that the driverless vehicles is like the "Turin Test" of the modern era. If only we can get a driverless car that does not run into things or kill anybody can we then finally say the car has an intelligent driver indistinguishable from a human driver? It would be a more fun thought experiment if not for the possibility of me getting ran over by one of these things lol.
Anyway, true autonomy, without any human intervention, may be further away than many companies have implied. If Cruise vehicles require frequent remote support even for test operations, full autonomy without humans may not be achievable in the near future.
"True autonomy" would not involve any human intervention or remote operator support during vehicle operation. The vehicles would be able to drive themselves without any human involvement, oversight or assistance. Could we agree on this definition?
The vehicles' artificial intelligence and capabilities would be sufficient that they do not require frequent "babysitting" or taking over driving duties by remote humans, as Cruise's vehicles seem to need according to the report.
The vehicles would be able to safely and reliably handle all driving situations and tasks without humans needing to co-pilot or effectively drive portions of routes. Any remote operator interventions would be very rare exceptions, not frequent occurrences.
Further, if we had arrived at true autonomy, companies would transparently disclose if and how much remote operator assistance is involved, rather than potentially obscuring technological limitations through human workarounds. Progress towards autonomy would be demonstrated through reduced reliance on humans over time, not maintained status quo involvement of remote operators behind the scenes. This would be legally sane and as a way of auditing them in some way.
Remote operators may effectively be "co-piloting" the vehicles and taking over driving duties in many situations. This calls into question whether the vehicles can drive themselves without human involvement.
The driverless car industry overall may be overstating progress made towards full autonomy without acknowledging the role of remote operators in keeping vehicles moving safely. We lack transparency about how much human support is actually needed. So far to me it looks like a mirage; it would appear that some vehicles can operate by themselves but only if we ignore the heavy babysitting programmers and the driver have to correct.
Companies are using remote operators to mask technological limitations and create the illusion of more advanced capabilities than have actually been achieved. This could mislead customers, investors and regulators about the true abilities. Not a good selling point and quite frankly fraudulent, if I might be so bold.
It seems Kyle Vogt confirmed the numbers from the NYT article. Cruise are so doomed, and if similar numbers apply for the other AV developers it would be a catastrophe for the industry, I think.
This too was incredibly surprising to me. And to many NY Times readers, based on the comments. At face value, it suggests the economics are still not there - at all - for these vehicles. The company will be under great pressure to disclose what these interventions are. To be their devils advocate, it could be that the vehicles have an intentionally sensitive warning system to which the appropriate human override is usually to click “proceed as planned.” Indeed, this could be a plausible strategy for acquiring fresher and fresher training data, if the data from these interventions are incorporated in subsequent training cycles. If so, I do think the company should only include unassisted driving tests in its safety statistics.
I find it funny that you suggest this is the 'dark secret at the heart of the entire driverless car industry'.
In my book The Future Normal, I profiled Robert Flack, the CEO of Einride, an autonomous electric freight startup in Sweden. He's very vocal about how full autonomy is some way off, if ever achievable – and indeed they made a big play about hiring a remote truck operator who manages a 'pod' of multiple trucks, ready to take over to handle last mile complexity.
Claiming / aspiring to full autonomy was a choice, not a requirement. This is why we need diversity within the industry.
From the NYT story: Cruise's human teleoperators "frequently had to do something to remotely control a car after receiving a cellular signal that it was having problems." We need regulatory investigations and private litigation to unpack the details of this. "Dangerous when used as intended" is what trial lawyers dream of.
Well this explains why they can't work when phone networks are clogged. Thanks Gary; I skipped over Rod's tweet, but then came to your blog off of Patrick Lin's Facebook, which I only saw because the EU's DSA forced Meta to put back in recommenderless newsfeeds...
If this is true, Cruise may also have been under-reporting interventions to the DMV.
Which it *looked* like they were, just comparing stories in SF Chronicle to the DMV's statistics on driverless vehicle events. But possible that there is some wiggle room in reporting requirements that means this was superficially OK.
1.5 operations staff per vehicle is another excellent example of the idea that automation is frequently not about replacing human labor entirely as it is about reconfiguring the labor process on terms more favorable to capital.
For those interested in the technology, strategy, and policy of self-driving cars, one of the best thinkers and writers is Brad Templeton. He has two recent fair-minded, balanced posts about the Cruise situation. More than technology, the most serious problem autonomous vehicle makers have nowadays is societal acceptance.
As information develops about the reported rate of human interventions, I would look to Templeton to boil down the facts and the implications.
This was quite a measured take, given your long-standing skepticism of self-driving cars.
There were signs in the past that Cruise was under pressure to deliver and was pushing things.
Waymo, by comparison, has been doing a lot more thorough work for a lot longer time. That, of course, is not a guarantee of anything. The world is complicated, and no tech will be ever be foolproof.
The hope is that Waymo will continue to scale cautiously, will carefully identify systematic issues, and work on solutions.
I am not really surprised that driver-less cars needs some human assistance. At the technology development or test stage it is quite normal. And it would be reasonable that remote assistance would be maintained for the full operational stage and even if the cars become more reliable.
It depends on the derivative, if they have x remote personnel per vehicle but that x is going down with time at a reasonable rate, they are in business.
Today's x number doesn't matter much, if it's on the path to be 0.1 in five years and 0.01 in ten years it looks great.
Any system does that information into function transformation as driving is. And needs a conscious observer for this to comply to the changing circumstances that are unpredictable and follow traffic rules. That'a a universal principle:
'Snake oil' keeps ringing in my mind. Those who keep insisting that LLMs are a step closer to AGI are selling the same snake oil, in my opinion.
Generative AI, regardless of its utility, is a step backward in the search for AGI. It has almost nothing in common with intelligence as we observe it in humans and animals. AGI will not be found under this lamppost. We need new models, new theories.
I wonder if Gary's too young to remember Spinvox, 20 years ago in the early mobile phone boom, who claimed to have technology that could transcribe voice recordings (eg voicemail messages) to text,
and burned through $100m of investor cash
and, it turned out, had never made it past the "secret call-centre-style companies of human transcribers" stage...
Plus ca change!
Gary, I kind of get the sense that the driverless vehicles is like the "Turin Test" of the modern era. If only we can get a driverless car that does not run into things or kill anybody can we then finally say the car has an intelligent driver indistinguishable from a human driver? It would be a more fun thought experiment if not for the possibility of me getting ran over by one of these things lol.
Anyway, true autonomy, without any human intervention, may be further away than many companies have implied. If Cruise vehicles require frequent remote support even for test operations, full autonomy without humans may not be achievable in the near future.
"True autonomy" would not involve any human intervention or remote operator support during vehicle operation. The vehicles would be able to drive themselves without any human involvement, oversight or assistance. Could we agree on this definition?
The vehicles' artificial intelligence and capabilities would be sufficient that they do not require frequent "babysitting" or taking over driving duties by remote humans, as Cruise's vehicles seem to need according to the report.
The vehicles would be able to safely and reliably handle all driving situations and tasks without humans needing to co-pilot or effectively drive portions of routes. Any remote operator interventions would be very rare exceptions, not frequent occurrences.
Further, if we had arrived at true autonomy, companies would transparently disclose if and how much remote operator assistance is involved, rather than potentially obscuring technological limitations through human workarounds. Progress towards autonomy would be demonstrated through reduced reliance on humans over time, not maintained status quo involvement of remote operators behind the scenes. This would be legally sane and as a way of auditing them in some way.
Remote operators may effectively be "co-piloting" the vehicles and taking over driving duties in many situations. This calls into question whether the vehicles can drive themselves without human involvement.
The driverless car industry overall may be overstating progress made towards full autonomy without acknowledging the role of remote operators in keeping vehicles moving safely. We lack transparency about how much human support is actually needed. So far to me it looks like a mirage; it would appear that some vehicles can operate by themselves but only if we ignore the heavy babysitting programmers and the driver have to correct.
Companies are using remote operators to mask technological limitations and create the illusion of more advanced capabilities than have actually been achieved. This could mislead customers, investors and regulators about the true abilities. Not a good selling point and quite frankly fraudulent, if I might be so bold.
It seems Kyle Vogt confirmed the numbers from the NYT article. Cruise are so doomed, and if similar numbers apply for the other AV developers it would be a catastrophe for the industry, I think.
https://www.reddit.com/r/SelfDrivingCars/comments/17nyki2/kyle_vogt_clarifies_on_hacker_news_that_cruise/?rdt=33497
This too was incredibly surprising to me. And to many NY Times readers, based on the comments. At face value, it suggests the economics are still not there - at all - for these vehicles. The company will be under great pressure to disclose what these interventions are. To be their devils advocate, it could be that the vehicles have an intentionally sensitive warning system to which the appropriate human override is usually to click “proceed as planned.” Indeed, this could be a plausible strategy for acquiring fresher and fresher training data, if the data from these interventions are incorporated in subsequent training cycles. If so, I do think the company should only include unassisted driving tests in its safety statistics.
I find it funny that you suggest this is the 'dark secret at the heart of the entire driverless car industry'.
In my book The Future Normal, I profiled Robert Flack, the CEO of Einride, an autonomous electric freight startup in Sweden. He's very vocal about how full autonomy is some way off, if ever achievable – and indeed they made a big play about hiring a remote truck operator who manages a 'pod' of multiple trucks, ready to take over to handle last mile complexity.
Claiming / aspiring to full autonomy was a choice, not a requirement. This is why we need diversity within the industry.
From the NYT story: Cruise's human teleoperators "frequently had to do something to remotely control a car after receiving a cellular signal that it was having problems." We need regulatory investigations and private litigation to unpack the details of this. "Dangerous when used as intended" is what trial lawyers dream of.
Well this explains why they can't work when phone networks are clogged. Thanks Gary; I skipped over Rod's tweet, but then came to your blog off of Patrick Lin's Facebook, which I only saw because the EU's DSA forced Meta to put back in recommenderless newsfeeds...
If this is true, Cruise may also have been under-reporting interventions to the DMV.
Which it *looked* like they were, just comparing stories in SF Chronicle to the DMV's statistics on driverless vehicle events. But possible that there is some wiggle room in reporting requirements that means this was superficially OK.
1.5 operations staff per vehicle is another excellent example of the idea that automation is frequently not about replacing human labor entirely as it is about reconfiguring the labor process on terms more favorable to capital.
Link to the Hacker News post: https://news.ycombinator.com/item?id=38145062
For those interested in the technology, strategy, and policy of self-driving cars, one of the best thinkers and writers is Brad Templeton. He has two recent fair-minded, balanced posts about the Cruise situation. More than technology, the most serious problem autonomous vehicle makers have nowadays is societal acceptance.
As information develops about the reported rate of human interventions, I would look to Templeton to boil down the facts and the implications.
His blog is called Brad Ideas.
https://4brad.com/
This was quite a measured take, given your long-standing skepticism of self-driving cars.
There were signs in the past that Cruise was under pressure to deliver and was pushing things.
Waymo, by comparison, has been doing a lot more thorough work for a lot longer time. That, of course, is not a guarantee of anything. The world is complicated, and no tech will be ever be foolproof.
The hope is that Waymo will continue to scale cautiously, will carefully identify systematic issues, and work on solutions.
I am not really surprised that driver-less cars needs some human assistance. At the technology development or test stage it is quite normal. And it would be reasonable that remote assistance would be maintained for the full operational stage and even if the cars become more reliable.
It depends on the derivative, if they have x remote personnel per vehicle but that x is going down with time at a reasonable rate, they are in business.
Today's x number doesn't matter much, if it's on the path to be 0.1 in five years and 0.01 in ten years it looks great.
Any system does that information into function transformation as driving is. And needs a conscious observer for this to comply to the changing circumstances that are unpredictable and follow traffic rules. That'a a universal principle:
Meaning - Information - Function - Learning ->
Knowledge - Consciousness - Understanding - Memory ->
Intuition - Thinking - Sensing - Feeling ->
DNA - RNA - Protein - Signal cascades ->
Nucleus - Wave - Quantum - Interaction ->
Protons/Neutrons - Photons - Electrons - Chemical Bonds ->
https://docs.google.com/presentation/d/1VCjOHOSostUrtxieZvOjaWuTNCT59DMF/edit#slide=id.p1