Happy 1st Birthday, GPT-4
Ten birthday observations
It’s been a big year, GPT-4! Ten birthday observations:
• We might be reaching a plateau in terms of sheer capability. Nobody has been able to beat it decisively. Places like Google and Anthropic have put in a lot of money trying. None succeeded; instead there tentatively seems to be convergence at GPT-4 levels.
• There has however been lots of progress both in discovering potential applications and in putting GPT-4 class models into practice.
• But putting LLMs into production is hard; most work to date has been preliminary. Companies are starting to temper their expectations. Many initial expectations were unrealistic.
• Bad actors, who may have lower standards for reliability, appear to be using them for cybercrime and disinformation.
• Some core problems have lingered, particularly hallucinations, instability and occasional nonsensical errors.
• OpenAI was here first, but there’s not a huge moat; many other players appear to be catching up. We can expect price wars.
• Litigation will be major issue; many questions about sources and the legality of those sources remain.
• Climate impact needs to be considered, especially as models continue to grow.
• Profitability remains in question. Big models cost big $, and getting standout results might cost huge $; licensing and legal costs may be substantial.
Gary Marcus feels pretty good about what he wrote in this earlier essay:
I think I rather wish the development of GenAI technology would slow down, passing through a consolidation phase, in order the necessary regulations, legal assessments, large public awareness, could really keep up.
As foundation models become bigger and bigger, accelerated by new iterations of GPUs and TPUs, LLMs will become more and more unpredictable. We’ll experience more productivity upheavals as model drift, “laziness” and irreversible results (from vendors and companies fine tuning LLMs) drain users of time and money. It may culminate with serious injuries or even death directly caused by LLM-driven autonomous machines, triggering bipartisan outrage and swift knee-jerk overreactions; we have watched this movie before (“it’s a wrap!”).