96 Comments
User's avatar
Jared's avatar

I say this a lot but it is worth repeating.

We will know AGI is close when the technology is able to handle legal research and analysis. The law has a tremendous advantage over every other field I am aware of when it comes to domain-specific AGI development: it has the best databases.

Platforms like Westlaw catalog every law, functionally all cases, law review articles, treatises, and so on. But not only is the raw i formation already all in one place, those platforms ALSO label all of it too. Effectively all the raw underlying information needed for legal research and analysis is already compiled.

Until we see legal research and analysis being handled at the “AGI level,” there is absolutely 0 chance we see any kind of broader AGI-type capabilities

Expand full comment
Ian [redacted]'s avatar

I'm in software development, and I think our industry is kind of AI-compatible because we have automated testing and can validate that an approach meets the stated goals.

To me, AGI isn't a helpful goal or a useful buzzword because AI in my job is like a slightly better electric drill. It's helpful in some ways, but you can't use a single electric drill to build a house.

Expand full comment
Kenneth Burchfiel's avatar

It seems (based on the feedback I've been reading) that if Gen AI were an electric drill, it would work most of the time, except sometimes it would drill 3 inches away from where you wanted it; sometimes it wouldn't drill at all; and on a few occasions, it would drill backwards into your hand.

Expand full comment
Larry Jewett's avatar

Wasn't there a movie that had a scene of a bot drilling into a person?

I think it was called "Botty Double"

Expand full comment
Paul Topping's avatar

AGI isn't a helpful buzzword because it doesn't yet exist. It will be useful because it won't just be a "slightly better electric drill". Because so many AI workers are lying about being close to AGI, it is natural to assume it won't be a game-changer when it finally does arrive. It will but you haven't seen it yet.

Expand full comment
Martin Machacek's avatar

AGI is a useless term, because it has no widely accepted definition. … at least not one I’d be aware of. Better tools (AI or not) are useful.

Expand full comment
AlexT's avatar

The Turing test works perfectly well. The real adversarial Turing test, though, not the cardboard version that BigAI likes to push.

Expand full comment
Paul Topping's avatar

You'd probably deny global warming because, hey, everyone has a different definition. AGI will never have a precise definition. How could it? That doesn't make it a useless term. AGI has been depicted in sci-fi novels and movies for at least a century. Each one is different but so what?

Expand full comment
Tek Bunny's avatar

Global warming is based on science and demonstrably exists. There exists no meaningful scientific theory of AGI and it demonstrably does not exists. You probably think vampires can exist because they are depicted in novels and movies.

Expand full comment
Paul Topping's avatar

False argument. You've swapped "existence" for "definition". You've lost the thread, mate.

It is quite possible that AGI, whatever the definition, will never exist. That's not where I put my money though. If you think you can make a case for that, write a post here. Good luck.

Expand full comment
Oaktown's avatar

You highlight a perfect example that makes the case for training symbolically enhanced LLMs exclusively on smaller, verifiably accurate data specific to a particular field or task. Seems to me that is the goal we should be pursuing; it's less destructive to the environment and favors true innovators over bean counting, insulated tech oligarchs competing with each other for the elusive Holy Grail of absolute power.

Expand full comment
Alex Tolley's avatar

A return to the "General Problem Solver" of yore? https://en.wikipedia.org/wiki/General_Problem_Solver

I think that an LLM interface coupled with symbolic AI might be the way to go. We can't go back to binary rules, but we need something fuzzier to determine a likely rule firing.

However, aren't the best lawyers and coders creative? How does real creativity get instilled in an AI? IMO, this is an important feature of human intelligence, and therefore, by implication, of AGI.

Expand full comment
Martin Machacek's avatar

Creativity is an ability to imagine things that have never been seen/done before, but yet are possible and based on current best knowledge. I’ve not seen any LLM produce anything mildly creative yet. Statistical pattern matching is unlikely to ever deliver creativity.

Expand full comment
Oaktown's avatar

I'm not a programmer, so I'm in over my head here.

The point I was trying to make was to develop LLMs trained only on subject-relevant and proven factual information which is enhanced by neurosymbolic AI, as opposed to the pursuit of AGI using LLMs to hoover up the entire internet, which includes a huge amount of vitriol, lies, hate, and general BS.

Couldn't LLMs trained for specific tasks also simulate creativity by finding patterns humans can miss due to preconceived notions and training? Isn't that what AlphaGo did when it won by using a move never before used by even the best players?

Expand full comment
Alex Tolley's avatar

I doubt there is enough legal documentation to allow the LLM to compress verbiage into useful speech. Legal language is also very stilted.

The way to do what you want is to use a basic LLM. Then tune it to use the legal documentation. The symbolic reasoning is still going to be hard to do, and I don't think you want to try to hand-code it either, so perhaps training it using reinforcement learning is the way to go here. The approach has been used by a number of teams to create LLMs for specific domains, e.g., maths, biology, etc. However, these approaches still lead to hallucinations. So, ideally, you need a way to ensure the legal information is not allowed to be incorporated into the LLM that "guesses" answers, but uses the correct approach to find the relevant cases, and integrates them to create the new documents. As you know, lawyers have been caught creating documents with AIs and fined. I have used an LLM to integrate with documents (RAG), but I haven't been that successful in getting great results. Also, the documents would be the whole legal database, which may be hard to access this way. I'm sure the likes of DeepMind are working on the needed approach, as Hassibis has said that a neurosymbolic approach is needed for getting good answers from documents, especially curated ones that are not internet sh*tpostings.

Bottom line, I am sure it is coming. There is a good business in legal work, and it is expensive. AIs to help with this as at least a first cut at doing the work, would no doubt be very useful. Have you researched what work is underway currently?

Expand full comment
George Burch's avatar

Discovery can be very knowledge domain specific. I think most legal cost is in discovery and hallucinations may be fatal.

Expand full comment
jibal jibal's avatar

No amount of refining of search engines (which is what "legal research and analysis" is) constitutes AGI. AGI requires autonomous goal generation--which is something we shouldn't want to implement.

Expand full comment
Larry Jewett's avatar

I thought we will know AGI is close when we can see the dots of its Ai's

Expand full comment
Diamantino Almeida's avatar

This as the feel of a few years ago with crypto, like the world would change dramatically, DAOS, smart contracts, among others, which I think as potential, but look where we are now. In the end it favored scams. Same story, different technology...it seems.

Expand full comment
Ian [redacted]'s avatar

VR (I still think it's cool), Crypto, Zoom's stock price during Covid, Canadian real estate, Dubai chocolate / Matcha everything all seem to follow the hype cycle curve (TIL that it's a Gartner idea) https://en.wikipedia.org/wiki/Gartner_hype_cycle

Probably AI will be useful for some stuff, just like I still use my Meta VR headset for 2-3 games, but the trough of disillusionment is a powerful thing once people see that the hype was mostly-BS. I'm looking forward to AI hype being over.

Expand full comment
Diamantino Almeida's avatar

I don't deny AI is in fact a technology that will dramatically change our lives. Still the way big tech and a few individuals painted, with AGI, and saying things that are not true about LLMs, making people believing is like a PhD, a friend, something you can trust. It's wrong in so many ways. This is scramble for power and divert resources to a technology, which I believe is amazing, but it's flawed. There are many other AI technologies and approaches much better.

Expand full comment
Brian Curtiss's avatar

Dubai chocolate is terrible. What a scam.

Expand full comment
George Burch's avatar

Not the full story though Bitcoin was supposed to have collapsed long ago and fraud and scams are always up front when new technologies occur or are made up.

Expand full comment
Diamantino Almeida's avatar

Yes, is not the full story for crypto, is a technology with many uses. There will always be opportunities, still the M.O., in AI feels very similar to the hype on crypto. Hopefully we will have regulation and someone will present us with a model that don't require such enormous data center.

Expand full comment
Brian Curtiss's avatar

I'm not sure how bitcoin generation at least will eventually be made less power intensive or easier. It's built to be the opposite over time.

Expand full comment
Ian [redacted]'s avatar

- David Graeber (or someone else) needs to update the book Bullshit Jobs for the AI era!

- I don't mind getting an AI-massaged bullet point list of technical options (during research) or things to look (during code review) at from a software developer team-mate as long as they have read them and interpreted them

- The problem with AI outputs *to my eye* is that I just gloss over them the way you accidentally forget that person you talked to at a party who checks all the boxes but is really empty and dull inside

Expand full comment
Chris Heinz's avatar

I like the Nvidia investment into OpenAI. Smells like some new flavor of Ponzi scheme to me.

Expand full comment
Jack's avatar

Or a barter economy. OpenAI can't/won't IPO to get cash for capital expansion but Nvidia is willing to make an exchange.

Expand full comment
Sugarpine Press's avatar

Not a disagreement, but a slight reframing. Two first-hand observations:

First. LLMs are able to increase the quality of output for corporate work, but require the same (or more) investment of time, attention, and thought (to ensure the final product isn't horse shit). They don't offer free lunches, just a little extra mayo.

Second. The smarter the user, the better the marginal quality gained for equivalent effort. The dumber the user, the higher the likelihood of mediocre output or horse shit. So, nothing really new there.

Expand full comment
Fabian Transchel's avatar

"The smarter the user, the better the marginal quality gained for equivalent effort."

I don't see how this could be true, because my experience with a sizeable sample of university students has been quite the opposite: AI acts as a gatekeeper/drug dealer for mediocrity. Brilliant students (and believe me, I know when I see somebody smarter than myself) refrain from using AI, because they so very clearly see that the net result is not positive.

In fact, I make an experiment with my Big Data course every semester. This winter term marked the first instance where the student's majority did NOT put AI on the ascending slope of the Gartner hype cycle, but on decline toward trough of disillusionment. I did not bias them toward where to put it, but I completely agree - I'm more surprised it took them so long.

Expand full comment
Sugarpine Press's avatar

Interesting. A different use case / application, perhaps? My use cases are corporate, and the value add was in research, summarizing, parsing, criticism, and editing. The human agent is already a domain expert by definition, and must make the AI do her will. Efficacy adjusted for G-factor.

If I speak to B.J., author of the DLU, I'm told LLMs are useless for advancing an original, creative project (and I agree). Worse, they are (as you say) a drug that rapidly attenuates cognitive ability.

University students, one hopes, are trying to learn how to think originally, creatively, and analytically, and I accept that AI can be antithetical to that project. However, I can assure you, 'original and creative' does not accurately describe day-to-day or week-to-week work in a corporate setting. (I'm not suggesting creative thinking doesnt occur in the corporate world, but there are whole provinces and varieties of work that are better described as repetitive with established principles and concepts at their center).

Expand full comment
keithdouglas's avatar

But why does that require a LLM or even an ANN more generally? I have colleagues who don't seem to get that "automation" is what computers are about generally, not some special category of tool or progarm.

Expand full comment
Sugarpine Press's avatar

I'm not claiming anything is "required," just that current LLMs have some marginal utility at common corporate work, for example, being directed by a domain expert to review or draft contracts according to a boilerplate and other examples. It's possible my expectations are so low, and my understanding of how LLMs function is just mature enough that I'm pleasantly surprised to discover any marginal utility at all. But in a corporate setting, I'm not able to claim there's none. I've observed firsthand there's a little. That's the extent of my data and my claim.

Expand full comment
Tony Gavin's avatar

GenAI is an accelerator and multiplier for smart people with domain expertize related to what they are using GenAI for. For those who don't fit that description (most people), GenAI is an impediment at best, and an outright danger at worst.

Expand full comment
--'s avatar

I would refer you to the recent METR study on AI-enabled productivity. Software engineers given AI coding tools believed AI would serve as an accelerator and multiplier, and indeed they reported the feeling of working faster. In reality, AI actually made them work slower.

The cognitive offloading and the engagement-optimized UX make people FEEL accelerated. But the constant need to critique and correct workslop is actually a hindrance.

Expand full comment
Tony Gavin's avatar

Until they get the issue of hallucination under control, that will continue to be a problem for any profession where precision is required. In our case, we are producing short-form content for more general consumption. Whilst accuracy is required, it's not a life-and-death issue, so there is a degree of tolerance that would not be permissible with other use cases. GenAI is very useful, but not nearly as useful (yet) as many would have us believe.

Expand full comment
Tom Rearick's avatar

One of my mentors in software development hired only what he called A hires. These were the best graduates of Georgia Tech. He paid them well, showered them them with free pizza, and locked them behind code-locked doors to protect them from Sales people with questions and interrruptions. He told me that if you introduce B hires, they ask questions and reduce the productivity of A developers. Besides, A hires prefer to work with other quick thinkers.

I know, this sounds like eugenics, but it is not. Software development is complex and benefits from really smart people...maybe not the most empathetic people, but people whose products enrich all of our lives.

I bring this up because AI is not even a B hire. It cannot be trusted. At all. It disgourges content representing the contributions of the bell curve (please don't call me a N.a.z.i). You do not want someone with an average IQ writing software. Yeah, AI can generate software but you will spend more time debugging the software of a AI than if you wrote it yourself.

Expand full comment
C. King's avatar

I think the difference you are rightly pointing to is between appreciating humanity and life in general, and appreciating levels and ranges of intelligence--as you seem to imply, a brilliant software writer can be a social dolt and a moral degenerate, though of course the question is always left open in the particular case.

But the question is, are we collapsing the whole idea of being human and the regard we have for it, with being scientifically/software-writer intelligent. I don't, do you?

Expand full comment
Tom Rearick's avatar

In 1995, cognitive anthropologist Edwin Hutchins said, “The last 30 years of cognitive science can be seen as attempts to remake the person in the image of the computer.” This viewpoint has only hardened in the last 30 years since that quote.

I have a pet theory that most developers of LLMs that confuse AI for natural intelligence have never parented infant children. You cannot raise an infant into an adult without marveling at the autonomy, creativity, originality, and independence of a naturally-intelligent being.

Expand full comment
C. King's avatar

Tom Rearick: I couldn't agree with you more about infant and child consciousness--and no one has to ask a baby to wonder.

Expand full comment
Micha Hofri's avatar

I admit to finding this complaint ridiculous. We, the world, has been exposed to GenAI for almost three years. I believe every technology developed by humans has taken an order of magnitude longer to produce an ROI. Consider its close relative: IT. I remember fretful articles in a variety of business publications during the 1970s, and '80s, even early '90s that having poured billions into information technology systems, and finding that companies can do many tasks with the information at their disposal than they could never do before, and do it much more quickly, and generate far better-looking financial reports, their ROI was negative or invisible.... Decades. Oddly, the change occurred in parallel with the Web entering our life; coincidence? I am not sure. The reasons for this long gestation? Most likely the very slow development of software engineering, which proved harder than expected, even harder than hardware engineering for a number of good reasons --- and now they have more or less merged.

GenAI has at least two difficulties when compared with other applications of computing: very limited, at best, understanding of the actual computing process, and the unattractive propensity to produce slop.

It will take a while....

Expand full comment
Trevor E Hilder's avatar

The reason 95% of efforts fail is that almost nobody understands how their organisation really works, or what activities create real value, rather than just being ritual performances that people enact when they go to work.

This is the same reason why ICT has not added as much value as was expected over the last forty years.

There is plenty known about how organisations really work, but almost nobody wants to know about it.

Expand full comment
JR's avatar

The big business investors chiefly assumed that AI would bring in automation at a mass scale -- or the chance to downsize while outsourcing productivity to silicon valley. That hasn't happened in the way they may have anticipated. For now at least, the joke's on them...

Expand full comment
Martin Machacek's avatar

I wonder what made them think that GenAI will be in any way useful for automation in general. Machine interactions with physical world are way harder problem than completing a sentence or creating a cute image. I’ve yet to see any advance in robotics or industrial automation that could be attributed to GenAI.

Expand full comment
Harold Toups's avatar

I’m not in favor of blaming technology for the workslop of incompetent lazy employees. Handed workslop by a peer, I’d return it with a nasty note … the first time. Unfortunately, too many businesses are handing a chainsaw to their employees with no instruction manual or training. They should expect damaged furniture and bloody limbs on the office floor.

Expand full comment
Graham Lovelace's avatar

'Workslop' is such a brilliant term. I'd also like to coin 'slopworkers' - an emerging group of employees or freelancers responsible for manually tidying, correcting, rewriting and otherwise perfecting error-strewn AI-generated content. Slopworkers are hidden from view and are never referred to. Their vital work is never recognised in order to avoid difficult questions over deep investment in unreliable AI 'solutions', and give the impression it is delivering huge productivity gains.

Expand full comment
Bruce Cohen's avatar

All workers will, in the future, have their legs cut off so they can fit in the Mechanicsl Turk cabinet.

Expand full comment
Terrance Stone's avatar

Not surprised. Anything AI genetates for me has to be rewritten. It's vslue is that it gives me a start and something to react to.

Expand full comment
C. King's avatar

Terrance Stone: Your comment about "something to react to" reflects a powerful point in cognitional theory that says: Contrast is illuminating.

Expand full comment
Lawrence de Martin's avatar

I have found that explaining technical problems to lay people can aid in finding a solution. Perhaps LLMs can serve this function- although it does pose an issue for the NDA.

Expand full comment
David Hite's avatar

Here's an alternative explanation: AI is moving so fast and changing so quickly that it's difficult for people to understand new capabilities and put them to use quickly. A lot of time is spent learning the latest and greatest, and that's hurting ROI. When the rate of change slows, ROI will go up.

Expand full comment
Fabian Transchel's avatar

Ha, the good old "adapt faster or be left behind". Look, ChatGPT will 3 years old in just a short month from now. If you look at the rate of change between Nov 22 and May 23, sure. But after that? Come on, the wall is very real and everybody can see that. We're in the equivalent timeframe of the Gigahertz race between Intel and AMD where even non-tech people realized that ever-longer instruction pipelines would not give us supercomputing*. Two decades later it's the same game, just scaled by 2 orders of magnitude.

Workslop is not an effect that will go away with more training. It's directly and imminently connected to human nature.

* I was just 12 f*** years old at the time and could see that just from looking at the (at that time, elusive) x86 spec.

Expand full comment
Earl Boebert's avatar

There's an irony here that's delicious for people of a "what fools these mortals be" persuasion.

The bulk of the hype and synthetic glory focuses on the generative aspect of LLMs, which is what they do so badly as to be limited to a very small number of business cases, and whose failures are so piquant as to get wide notice. Which may end up killing off the whole movement, which in turn may be bad thing.

The "AI will write code for you. AI will write legal briefs for you." kind of hype obscures what I have found LLMs to be really quite good at, which is checking human work. So instead of saying "write me an essay on applying Bayesian reasoning to forensics," say "Here's an essay on the application of Bayesian reasoning to forensics. Tell me what it misses and what it gets wrong." The latter form of prompts plays into the pattern recognition strengths of these beasts. Just don't tell the robot you wrote it, or it might fawn and drool instead of criticize.

I think you will find, as I have, that it is much easier to asses a robot's critique of something of yours than trying to critique something the robot generated out of thin air. The reason is pretty obvious. In the first approach you have already studied the problem space and can relatively quickly and efficiently catch robostupidities. In the second approach you and the robot are learning together, and you may be just as ignorant and error-prone as it is.

So I argue that the ROI is poor because the proper role of the robot is to increase the quality of the human output, whereas foolish and lazy humans are instead using the robot to increase quantity and end up drowning in roboslop.

Expand full comment
C. King's avatar

"Roboslop." Love it.

Expand full comment
Tony Gavin's avatar

GenAI is excellent at critiquing work. We create a lot of AI-generated content for the web. Part of the workflow involves using a different model than the one that generated the content, which critiques the output for prompt compliance, factual accuracy, citations, structure, and other factors. After all that, it's handed off for human review, and we still find errors and inconsistencies. GenAI saves time and leverages the efforts of intelligent people with domain expertise. The problem is that most users are lacking in those areas.

Expand full comment
Guidothekp's avatar

ROI on Generative AI is poor because the usage is not organic.

The field suffers from ingenuity in selling the tech. Perplexity, with its 30+ billion valuation, comes up with uses cases such as finding deals and gathering leads. Look up why they wanted to buy Google Chrome. 2010 called -- it wants its "stories" back.

Why isn't the uptake organic? Because tech bros don't have time. They believe a baby if forced to learn can solve dynamic programming problems after two days. There is a movie on Netflix called "Serious Men", which reflects this without the makers realizing it.

Organic uptake takes time. The users will figure out what a tech is useful for. The well fed tech bros can pontificate but it will make no difference. As an example, look at social media. It went from a tool to post your breakfast pictures to citizen journalism to organizing movements to talking directly to the politicians and more. This tool might have been designed as a way to spy into users but it is users who are running the show now. All these took 15 years +.

The marketing of Gen AI needs to be far more subtle. By coming on strong, the bros have ensured that every failure will be magnified and celebrated.

Expand full comment
Marc Schluper's avatar

In 2006 people were laughing at Google because it had spent $1.65 billion for a place where people put their vacation videos, YouTube. The ROI zealots felt so right.

YouTube's estimated 2024 profit is $29 billion.

It's way too early to judge GenAI. We just opened the door. There is a vast unexplored field.

Expand full comment
Toiler On the Sea's avatar

Youtube was WIDELY popular upon release. Demand for its content did not have to be forced at all. Not the same for AI.

Expand full comment
Jonah's avatar

Conversely, in 2012, Google purchased Motorola Mobility for USD 12.5 billion and sold it two years later for USD 2.9 billion. Or on the "smaller" end, ITA Software, for around USD 700 million, with an onerous purchase requirement that obligated Google to license the software for five years. They ended up shutting it down because it had made so little money.

They have made good acquisitions, too, of course, like the one you mentioned, but I think that large companies illustrate a principle that has been much discussed in an individual context: failing upward, maybe with a bit of the Peter principle included. These two ideas combined, as you probably know, would suggest that someone starts with some degree of actual skill that lets them get their foot in the door, and initially rises based primarily on their skill, but then they reach a level of incompetence (Peter principle), after which their exaggerated confidence lets them continue rising well past the level where they are actually contributing.

What I think often happens with large companies is similar: they originally have an excellent idea and grow quickly from a small company until they reach a point where they can no longer provide much more value to customers. However, they still retain their extreme confidence, so they keep growing, since this appeals to investors. Then, like an incompetent manager taking credit for the successes of their underlings, they buy out smaller companies that are still in the "good ideas" stage, less through special discernment than through accumulated wealth, and thus cannibalize some of those companies' growth for their own.

Expand full comment