About that OpenAI “breakthrough”
Will OpenAI’s Q* change the world?
[Newscaster]: We interrupt your Thanksgiving to bring you this bulletin.
Yesterday at The Information, in the continuing drama surrounding OpenAI:
Later, at Reuters, further developments:
The technique in question appears to be something OpenAI is calling both ominously, as Q*, pronounced “Q-Star.”
Get it? Perfect grist for anyone who wants to freak about the Death Star killing us all. (In reality, though, it seems to be named in reference to a clever but not particularly ominous AI technique known as A* that is often used to power character movements in video games..)
The board may well have genuinely have been worried about the new techniques. (It’s also likely they had other stuff on their mind; according to the same story in Reuters, the handling of Q* is allegedly “[just] one factor among a longer list of grievances by the board leading to Altman's firing, among which were concerns over commercializing advances before understanding the consequences”.)
§
Me? I am pretty skeptical; I was only half-kidding when I wrote this:
§
Part of the joke is that I don’t actually think Bing is powered already by the alleged breakthrough. A single fail on Bing a few months after said putative breakthrough isn’t going to tell us much; even if it were real, whipping Q* (or any other algorithm) it into production that fast would be extraordinary. Although there is some talk of OpenAI already trying to test the new technique out, it wouldn’t be realistic for them to have entirely changed the world in a matter of months. We can’t really take Bing 2023 to tell us anything about what Q* might do in 2024 or 2025.
But then again I have seen this movie before, often.
OpenAI could in fact have a breakthrough that fundamentally changes the world.
But “breakthroughs” rarely turn to be general to live up to initial rosy expectations. Often advances work in some contexts, not otherwise. Arguably every putative breakthrough in driverless cars has been of this sort; somebody finds something new, it looks good at first, maybe even helps a bit, but at the end of the day, “autonomous” cars still aren’t reliable enough for prime time; no breakthrough has gotten us over the threshhold. AV’s still aren’t general enough that you can just plop down a car that was tuned in pilot studies in Menlo Park, SF and Arizona and expect it to drive gracefully and safely in Sicily or Mumbai. We are probably still many “breakthroughs” away from true level 5 driverless cars.
§
Or consider what was touted at OpenAI as an extraordinary breakthrough in 2019, when they launched a slick video and blog post about how they’d gotten a robot to solve a Rubik’s cube::
To many, the result sounded amazing. VentureBeat gullibly reported OpenAI’s PR pitch wholesale; “OpenAI — the San Francisco-based AI research firm cofounded by Elon Musk and others, with backing from luminaries like LinkedIn cofounder Reid Hoffman and former Y Combinator president Sam Altman — says it’s on the cusp of solving something of a grand challenge in robotics and AI systems.”
Me being me, I called bullshit, slamming OpenAI on Twitter a few days later:
Probably the most relevant bullet point on the right (at least for present purposes) was the one about generalization; getting their algorithm to work for one object (which, cheesily, turned out to be a special Cube instrumented with sensors and LEDS, unlike a Rubik’s cube you’d buy in a store) in carefully controlled lab circumstances hardly guaranteed that the solution would work more broadly in the complex open-ended real world.
Notwithstanding my concerns, OpenAI got a tremendous amount of press, and maybe some investment or recruiting or both off the press release. Looked good at the time.
But, you know what? It went nowhere. A year or two later, they quietly closed the robotics division down.
§
The thing that cracked me up the most about the Reuter’s piece was the wild extrapolation at the end of this passage:
If I had a nickel for every extrapolation like that—today , it works for grade school students! next year, it will take over the world!—I’d be Musk-level rich.
§
All that said, I am scientist. The fact that a lot of past performance has been overhyped no guarantee that every future advance will fail to pan out. Sometimes you get nonsense about room-temperature superconductors (such narratives always play well initially), and some things really do pan out.
It’s all still an empirical question; I for one certainly don’t yet know enough about the details of Q* to judge with certainty.
Time will tell.
But, as for me, well… I got 99 problems I’m worried about (most especially the potential collapse of the EU AI Act, and the likely effects of AI-generated disinformation on the 2024 elections).
At least so far, Q* ain’t one.
Gary Marcus has been resisting hype for three decades; only very rarely has he been wrong.
From an ML perspective, LLMs are an amazing achievement. Who would have expected such impressive (from the perspective of an external observer) performance from such a simple NN model trained with vast amounts of (relatively low-quality) data, and vast amounts of compute?
From an AGI perspective, however, if your objective was to build literally the *worst* AGI possible (relatively weak cognition (lots of data plus statistical induction plus analogical reasoning, but no deduction or abduction), extremely shallow (if any) internal model of the physical universe, appallingly misaligned with aggregate human preferences, no understanding of how it actually works inside, let alone mathematically well-founded), then you couldn't have done a better job.
I await further details of Q* with trepidation...
What's really wrong at Open AI is that all female board members are out, whereas all the men involved still have roles. Adam D’Angelo and Ilya Sutskever were 50% of one faction. Altman and Brockman 100% of the other faction. Net result…. Although technically Altman, Brockman and Sutskever are no longer on the Board, they are still in place. And we got two new rich white guys, Larry Summers and Bret Taylor. I’ve lost track of the score but I think we’re around -12 for Human Intelligence vs 0 for AGI.