48 Comments
Feb 15Liked by Gary Marcus

These articles/statements about programming a computer using English versus a formal programming language all start from the assumption that English is a superior tool. I think that's because we (English speakers) are all really familiar with it because we've been using it for our entire lives! But if you spend any amount of time thinking about it - English is a terrible language! We humans are terrible at communicating with each other and we think a computer is somehow gonna be better? How many times have you said or heard someone say, "...you just don't understand what I'm saying?" That's an English language "programming bug" between two native English speaking computers and it happens ALL THE TIME!

Expand full comment

The uses of any human verbal language are much, much different than the systematic strings of coding instructions used to program a computer. So different that referring to computer programming "languages" is a misnomer. Computer lexicons don't function in any way analogous to human verbal languages, which are much, much more elastic.

That said, your point is correct. As verbal languages go, English is one hell of a spaghetti code. (To resort to an idiomatic metaphor- which no computer could ever properly comprehend, even if it were to parrot the phrase "spaghetti code" with passably correct usage as part of an AI-written essay.) But all human verbal languages rely so heavily on connotative meaning, metaphor, descriptive modifiers, specific context, idiom, etc., and verbal languages are used by humans for so many other purposes than giving orders or obeying orders (which is what computer programming is all about) that I doubt that even a relatively more logically consistent and economically structured language (like Spanish) would offer much help to the strict and unvarying literalism of machine intelligence.

Expand full comment

This got me wondering - are there any studies exploring which "human languages" (i.e. english, spanish, japanese, etc..) is better/best suited for machine interaction? Or is the CS world so english-centric that this study wouldn't be statistically possible and/or rigorous enough?

Expand full comment
author

Best results currently are in English because these particularly techniques are very data dependent and English has the most data.

Expand full comment

Yep. Cultural hegemony has its privileges, which have no relation to intrinsic logic merit. As inaugurators of the digital age, whether as inventors, developers, innovators, or the initial consumer market, Anglophones got there fustest with the mostest. And so it came to pass that English became "the language of the Internet."

Glorious English. In all its loose-ended, mixed-etymology sloppiness and ambiguity. Silent vowels, silent consonants, alternate pronunciations of the same spelled letter order, 26 letters for 40 phonemes...that isn't the half of the cybernetic noise glitches encoded in the English language. There are also irregular verbs, prepositional criteria, idioms, neologisms, homonyms (to/too/two-! Not even the same parts of speech!), misleading cognates, subject-object confusion...that's just an incomplete list of "problematic features" at the structural level of phonetics, grammar and syntax. Before semantics.

I sympathize with almost anyone having a difficult time with learning to read, write, and spell English. The English language is on a power trip. You can't lodge a protest against English on the basis of its numerous violations of logical consistency. You just have to get used to the conventions by practicing them.

Expand full comment
Feb 15Liked by Gary Marcus

Reminds me of the ol' DWIM (Do What I Mean) instruction and, augmenting the GoTo, we have the very helpful ComeFrom. Not exactly English but close enough....

Expand full comment
Feb 15·edited Feb 17Liked by Gary Marcus

Two things:

1. This is exactly the point. The inherent randomness of GenAI is new to IT. I told people in a meeting today: "the newness is that IT always wax essentially explainable and predictable, even big data analytics. But GenAI's essence is that it is essentially unexplainable and unpredictable. This will be something that will be hard for people, as they will tend to assume the old.

2. Dixit Perlis: “When someone says, “I want a programming language in which I need only say what I want done,” give him a lollipop.” (repaired after Scott's helpful correction below)

Expand full comment

The lollipop quip is by Alan Perlis, not Knuth. Perlis is perhaps not well remembered now, but he was the first Turing Award winner.

Expand full comment

Indeed. I should have checked.

Expand full comment

What we gain, however, with LLM, is a very large amount of flexibility and approximate reasoning, which did not exist before. As these systems get used more, a large corpus of data will get developed that will add the right context and invoke the right agents. There will be pipelines for verification as well. The current approach is very promising, and there are zero alternatives.

Expand full comment

So, the problem is this: if your model is by definition 'approximating', it will also approximate the call to any non-approximating stuff in the background. We are seeing now early steps (GPT4 producing correct python code for very simple arithmetic) but there is no reason to assume that when this gets a bit more complicated the route via that code will become reliable. More reliable than without — certainly — but it doesn't solve the problem. https://ea.rna.nl/2024/02/07/the-department-of-engineering-the-hell-out-of-ai/

The same is true for 'pipelines for verification.

Both by the way are part of the 'engineering the hell around the fundamental limitations of LLMs'. This means for me that I suspect we will get some effective ('satisificing') tools out of this, but there is no chance in hell we'll get anything resembling AGI. Not even if scaled a thousandfold https://ea.rna.nl/2024/02/13/will-sam-altmans-7-trillion-ai-plan-rescue-ai/ (sizing really is not a cure-all).

Finally, the idea that we are getting a lot of flexibility is both true and not. At the user interface we get a lot of flexibility (because of the natural language in/output, but at the expense of ease of 'programming') but take one step back and you are looking at extremely large digital systems which have the property that they are very hard to change. For instance, all the fine-tuning and filtering to get GPT 'helpful, honest, and harmless' is making it less flexible (it becomes a bland 'middle of the road').

So, I suspect we will get — with serious engineering and complex architectures — useful tools. There is indeed some promise. Not materialised yet, but there is promise.

Expand full comment

It is a little premature to talk about AGI. What we want, for now, is reliable assistants. There are lots of problems to solve till we get to AGI.

It is true that that we cannot produce software with plain English. The plain language part of interfacing with a chatbot should be as short as possible, and it should have hooks into real tools and simulators as early as possible.

So, the hope is to have an agent that is skilled at figuring out what the users want in plain language, and is smart enough to know which trusted tools to pick to get the job done, and how to check if it did the right thing.

Expand full comment
Feb 15Liked by Gary Marcus

I think about LLM (especially coding via LLM) as writing down your request, sliding it under a door, and getting a paper or book (or program) back. You have to work to verify that the output meets your expectations, you have no idea how it produced that output, and the same request slipped underneath might result in a partially or entirely different result.

(We had a similar fad in the late 90s/early 00s about code generation. You'd do some light work in a DML, run a code generator that had a template to turn that DML into a _slew_ of code. The difference then was that it was deterministic and you could read the template or the code generator. This is so much worse in nearly every respect.)

Expand full comment
Feb 15Liked by Gary Marcus

It took too long for this article to get to the point I was thinking as soon as I started reading: natural language is, by nature, way too imprecise for serious work. Code written in English will inevitably make unwarranted assumptions, based on statistical likelihood, that are difficult to de-program. Getting the code to do exactly what I want will be like if my spell check repeatedly corrected the spelling of my last name from “Cooke” to “cook,” without my being able to go back and override it.

Expand full comment

Can't resist: D. Knuth. "Literate programming is a methodology that combines a programming language with a documentation language, thereby making programs more robust, more portable, more easily maintained, and arguably more fun to write than programs that are written only in a high-level language. The main idea is to treat a program as a piece of literature, addressed to human beings rather than to a computer." (I know, not quite the same as dictating your code in English, but I would bank more on this approach in view of the current state of the art).

Expand full comment
Feb 15·edited Feb 15Liked by Gary Marcus

The natural language for humans is, well, natural language.

The language for computers is code. It's more precise and structured, while being less flexible and robust wrt fuzzy shades of meaning.

The friction in computer programming for 50 years has been that difficulty in translating human intent into computer programming language.Gen AI may now bridge that gap by making more natural language inputs suffice to direct the rest of the process, in effect compiling from natural human language to computer code via code generation. Maybe a more literate yet also precise language might emerge in the gap to assist AI's role. However, the gap between vague human intent and precise instructions for a computer remains.

Expand full comment
Feb 15·edited Feb 15

The language for computers is not simply code, it's machine code that gives instructions to the CPU, which in the binary computers we use is represented in binary. We have already bridged the "gap between human intent and precise instructions for a computer", because we long ago translated the language of computers, which is machine code that gives instructions to the CPU, into a form we can work with. How many programmers do you know who program in binary, which is the only language our binary computers understand without translation?

Every form of programming that is not done in binary, in practice all of it, is a translation intended to make it easier for humans to understand. Even hexadecimal programming, which is closest to writing machine instructions in binary, is inherently a translation of base 2 into base 16 to make it easier for us, but there is a reason computers use the reliable base 2 that can accurately correspond to the high and low electrical signals received physically.

Asm, of course, is merely a symbolic representation of machine code, a "translation" if you prefer. Not to mention high level programming languages, all of which such as C++, C#, Java, etc are many, many layers of abstraction removed from machine code, and are easier by far for us to understand. They must all ultimately be translated to a lower level for our computers to understand, however, and hence why compilation is necessary and a compiled program can only be understood by us through a disassembler or in base 16.

The "friction" has already been solved, the gap already bridged, by the pioneers of computing. There is no reason to assume natural language would be better to program with than the easy-to-understand high level languages we have that anyone can learn, or that they wouldn't result in a far messier translation into binary machine code, especially if the natural language AI code output is in a high level language anyway—everyone knows what happens if you Google translate a natural language five times among other languages, into Chinese, then back to English.

Gen AI is ultimately so much more limited in use than we're told, this is a solution in search of a problem that doesn't exist. Natural language has no place in programming, nor does any existing AI, which can only make a mediocre programmer lazier and perform worse at the task while being of no use to a skilled programmer. Both are basically only desired by non-programmers who wish they could program, but don't want to learn.

Expand full comment

"The "friction" has already been solved, the gap already bridged, by the pioneers of computing."

No, as a software engineer I can tell you that the problem has not been solved, because we are constantly improving programming languages and techniques. There are gaps in programming and software dev productivity to be bridged still.

"There is no reason to assume natural language would be better to program with than the easy-to-understand high level languages we have that anyone can learn"

It's far easier and more accessible for someone to express their intent in human language than in a programming language they have to learn.

I disagree that natural language has no place in programming. Design docs, tests, user stories, and coding comments and documentation use natural language for a reason - for our human benefit. One small step to have human-readable code. An expression or representation that both humans AND computer can understand is one way to bridge the human intent to computer operation gap.

Expand full comment

I recently had a discussion (argument?) with someone on Reddit about the false promise of programming in English. They were claiming that LLMs would revolutionize this. As you say here, the problem isn't translating the English to code. Sure, perhaps LLMs can help with that. The real problem is that people can't describe useful, non-trivial programs in English. That's not going to change any time soon as it's a human limitation. To do it properly requires a conversation that contains many exchanges like this one: Human: "... and then I want the program to print the list." Programmer: "What if the list is a million pages long?" Human: "Well, then I wouldn't want it printed out but saved as a text file."

What we really need is an AGI that can take the human's original problem description and ask questions about it, and fold what it learns into an eventual solution. This is what a software analyst does and the good ones get paid big bucks to do it. LLMs aren't close to doing this.

Expand full comment
author

💯

Expand full comment
Feb 15Liked by Gary Marcus

Imagine the pointy-haired boss explaining what he wanted the computer to do, to Dilbert.

Expand full comment
Feb 16Liked by Gary Marcus

Absolutely my first response here is "Dude, have you _seen_ English?"

But close behind is wait, whose English?

Expand full comment
Feb 16·edited Feb 16Liked by Gary Marcus

"It’s because formal languages make it much, much easier to specify what you want" - that is precisely why we have invented the whole mathematical notation, because natural language is imprecise and ambiguous. To precisely specify something in natural language would require a lot more text than a formal language and the uncertainty whether the other party understood you correctly remains because there is no formal description of natural language and each individual has slightly different understanding of what each expression means, an understanding that we acquire through personal experience which varies for each person.

Expand full comment
Feb 16Liked by Gary Marcus

Exactly. Trying to specify, for example, a video codec in plain English is plainly futile.

Expand full comment
Feb 15Liked by Gary Marcus

"My trust in the coding infrastructure was essentially absolute. " ^100. This looks like another assault against the truth where the truth takes a back-seat behind the convenience of the AI.

Expand full comment
author

Well-said

Expand full comment

The problem is that there is so much work that is very hard and very expensive to code up. For problems where verification is easy, chatbot-based approaches will work very well.

Expand full comment

English has many levels, just like World of Warcraft has like 70 or 100 levels. A computer won't understand a quaternary definition of a word in an esoteric context, unless it has been trained on that very instance. In many ways, the intentional obscurity in-jokes code-speak is what makes English organic, rather than standardized, to keep machines from learning too much semantics. A PhD might help one get further in leveling up. Or reading a lot.

Expand full comment
Feb 23Liked by Gary Marcus

If English were a great tool for precisely expressing behavior then contract law wouldn't be a thing.

Expand full comment

Great stuff here. A big part of the problem, I think, is the inherent ambiguity in language. A word points to an object in the world, but it is not equivalent to that object. Code, in contrast, points to fairly discrete digital objects (1s and 0s, pixels, etc.). Language can't be reduced to 1s and 0s, and the *meaning* of images can't be reduced to 1s and 0s, because all that stuff is ultimately ambiguous!

Expand full comment
Feb 16Liked by Gary Marcus

It’s nuts how quickly so many are eager to forget the valuable lessons around software development. Back in the early aughts at Carnegie Mellon, Applied Math/Computer Science majors didn’t spend much time learning programming languages at all. The first two meaningful computer science classes were Fundamental Structures of Programming I & II. Most who graduated from that program could learn a programming language in a day or two and write elegant code because the syntax was never been the most important part, it has always been the careful structuring of the code. Second important principle was that the further away you got from the machine’s most native code (machine language), the more likely complexity would start to creep in. Assembler language was the next best thing, but after that developers are subject to the bugs of the various compilers or interpreters. The English language as a coding language has to be so far up that totem pole as to make writing code a monumental effort just to get even the most basic things right. Maybe someone will figure it out, but it feels like it cannot possibly lead to better software being developed, not to mention that those too lazy to learn proper programming languages are likely not the sorts of people we’d want developing code anyway 😉

Expand full comment
Feb 16Liked by Gary Marcus

I am neither a language expert nor a programmer in the strict sense. I am 'only' using Python and R for scripting as a researcher. But from that perspective, it seems exactly right to me that the point of a programming language is to make the computer do exactly what it is told. It would break everything if the fuzziness of natural language would be introduced into the arrangement, starting with the reproducibility of analysis workflows.

Expand full comment
Feb 16Liked by Gary Marcus

"We have apparently just entered a new and different and frightening era, in which debugging involves not only fixing our own mistakes but coming to terms with machines that may or may not uphold their end of the bargain."

The importance of this cannot be overstated. Computers do what we tell them to do and they do it reliably. When I type 2 + 2 into calculator and hit enter it will always output 4, there is literally 0 chance that "hallucination" will occur and then suddenly it's not 4 anymore.

Expand full comment