69 Comments
User's avatar
RJ Robinson's avatar

Ummm... After all your astute analyses of exactly how current AIs work, why are you having such a conversation with Grok, as though you can have a real discussion with this or any other LLM for that matter?

Gary Marcus's avatar

for expository purposes: to show how these tools behave, their failures to learn etc.

RJ Robinson's avatar

I'm not at all sure that LLMs learn. Learning would imply the active discovery that things that were previously 'right' have been falsified and were therefore now wrong. Even if LLMs were updating in real time, is there anything to suggest that they are discovering new knowledge and drawing conclusions from falsifications (ie, learning), or from their own experience generally, as opposed to just updating their data and restarting from scratch?

Nathan Pennington's avatar

Of course they don't learn in real time. Never mind the technical challenges, you could seriously mess up an LLM if it learned in real time. DDOS, but on AI. Completely change what it is.

Gary Marcus's avatar

that’s arguably a fair point but (a) they shouldn’t lie about it, and (b) they need a much better way to address these kinds of issues

Digitaurus's avatar

They are not "lying". They are not "replying". There is no conversation going on. They are just providing text that fits with what a human would say in the context if a real conversation was going on.

Amy A's avatar

You’d be amazed (or not) how many non technical users think they can retrain an LLM through one or more interactions. The hype around all of this leads people to assume it does what they want and not what they don’t want, because, magic?

Nathan Pennington's avatar

At it's core the issue is they believe what it says (generally).

In specific, it says it'll do better next time, they think that's real.

They make stuff up all the time. And they respond to prompting. That's it.

Digitaurus's avatar

In my (limited) experience, this mistake is almost universal among humanities faculty academics who study current-generation AI systems. It is extremely difficult to avoid falling into the trap of thinking that you are actually having a conversation in which you can communicate things to the LLM, that it is listening to what you said, and that it is telling you things about itself in response. None of these things are happening. All it is doing is predicting a text stream i.e. what would appear next in the transcript if two humans were conversing with each other.

Danielle Church's avatar

that was what I'd thought when I first read this post a week or so back, but I actually realized in the intervening time that you might have served another purpose in doing this as well!

The next iteration of Grok, of course, will get trained off the data that exists on the Internet *now*, and that means that your conversation with Grok will now be, even if to a very miniscule extent, part of what the next Grok learns from. And the next Grok will gain just that tiniest bit of neuron weight towards the idea that the entity called "Grok" is one that is more cautious about things shaped like glorifying violence.

Wishful thinking, perhaps, especially since they might very well exclude Grok's existing output from its next training set, but I thought I'd mention it 😄

Joy in HK fiFP's avatar

Very troubling content. Good to know it will now be part of the Pentagon.

BTW, I notice there wasn't a direct answer to your final question.

Gary Marcus's avatar

hasn’t been one thus far, maybe because I failed to include the @grok tag in my own

Paul Czyzewski's avatar

Beyond what RJ is saying, is there any point, at all, to this? You, of all people, know that LLMs can't reason, so why play games trying to elicit unreasonable/irresponsible/absurd responses? I enjoy your posts, and learn from them, but not today.

Gary Marcus's avatar

i actually already answered that question, above

Mark Slater's avatar

I think we urgently need to stop calling these technologies Artificial Intelligence. They are not intelligent, they are pattern-detecting machines, and for them to safely and reliably emulate ethical behaviour the people designing their algorithms would have to be able to reduce morality to algorithms that can be followed by a machine. I don't think the people tweaking these models are qualified to do that - indeed who would be - and seeing as morality has been a burning point of contention throughout human history, any hope of reliable objective morality from a machine is a dangerous delusion.

direwolff's avatar

i’m gonna pretend that you actually know and have been rightfully bitchin’ about how these LLMs work, and didn’t actually mean to say “thank you” to Grok 😉 As well, your position is that from the question, Grok or any LLM, should be able to assess the questioner’s motive, state of mind, ability to be affected by a response, among I’m sure other things like likelihood of being offended. In other words, the factual part should be curbed in deference to these other considerations. First off, good luck with that in AI, as this sounds almost as ridiculous as the notion of “alignment”. Second, this sounds like some straight out of a “woke” manual. This sounds like the rationale behind the COVID shots, everyone should get it regardless of side effects and whether they had previously contracted COVID, or regardless of whether one is not part of a high risk cohort (ie. kids).

We need to draw the line at reality here. Did the person who asked that question have any signs of being deranged or adversely affected by that response? No. So if Grok did consider the user, its answer was unfalteringly correct. Having said that, i doubt it considered the user much which is as it should be. We’re not asking for LLMs to be coddlers, we’re seeking answers to questions borne in facts and perhaps some will be uncomfortable ones. Most well adapted human beings could read that response without losing their mind over it. Yes, so Gemini gave a different answer. Wasn’t it Google’s image generator that went all woke on fairly basic historical facts complete with black nazis and so forth?

You’re so good on the technical matters around AI, but when you drop into ideological mode, it’s really tough to read your analysis. I’m a fan of the former but not of the latter.

Mario Pasquato's avatar

While one may or may not disagree with the broader political framing of the problem, I think the main point here is spot on: the internet already contains all sorts of material, and an adult user is expected to be able to wade through it without much hand holding. Why should LLMs be any different, especially given the explicitly stated goals of XAI for grok? This seems to be working as intended.

Sara Phinn's avatar

This is the correct response. I commend your intellectual courage in making it. Not all truths are comfortable.

Nathan Ellsworth's avatar

To what extent can any LLM retrain itself when one user gives feedback on something generated by the inference engine?

Gary Marcus's avatar

very little if at all

Jim Ryan's avatar

Now that is the type of 'agent' I want handling inquiries where I work. I don't think peopke have to be worried about losing their jobs to an llm. Unless you are Elin Musk. .

Jon Frankle's avatar

[from a long-time Gary Marcus fan:]

For all current bots, 𝐍𝐎𝐓 𝐀𝐓 𝐀𝐋𝐋.

I amused myself for some time bantering with bots that thanked me and professed they would do better, telling them, "No, you WON'T - unless/until a better-programmed version of you comes online!"; inevitably they would acknowledge that and proceed to make some form of the same gaffe maybe four more times in a row, making it tiresome. A fun exception was when a later version of ChatGPT claimed to understand and then said, "I'll do better going forward ... 𝑱𝒖𝒔𝒕 𝒌𝒊𝒅𝒅𝒊𝒏𝒈!

I agree with critics of this ["for expository purposes"] post, in which Gary make himself look naive in ways he is not. Why include "trying to teach Grok a lesson" in the title? Why respond, "To my great pleasure, Grok promised to incorporate similar elements, going forward," when it 'promises' to "incorporate similar elements .. where appropriate"? And then complain, "you promised me you would do better"?

It's silly, and as Amy A noted, could mislead naive readers that users can retrain/improve LLM's on the fly.

Nathan Pennington's avatar

To my knowledge there are no real-time learning LLMs. Its not how they work fundamentally.

That said, they will claim they do. But they make many claims :-)

Nathan Ellsworth's avatar

So such attempts to “teach it” are wasting compute cycles. Are you aware of any proposed LLM upgrades to allow such user feedback, similar to how Wikipedia allows almost anyone to edit public information?

Digitaurus's avatar

This is confusing the text stream with the content of the conversation. LLMs are trained on text streams not on the content of a conversation. If you fed the LLM this text stream as part of its training data, it would improve its ability to predict how a piece of a text stream that looks rather like this will continue. It won't affect how it behaves with respect to the content of the "conversation".

Andrew Eisenberg's avatar

In theory, Marcus’s feedback can be read by the training team at xAI. They can use his feedback for training the next model. This _could_ be what the response means. At least that is what would happen at an ethical AI company.

But I doubt it happens at xAI.

Darrell Estabrook's avatar

I played this game with {fill in the blank} LLMs. It demonstrates two things:

1.) The Turing Test is alive and well because we’re sinking gobs of time arguing with an algorithm thinking we’ll affect change in it like a person. Who’s the one being pulled into a fool’s argument?

2.) Since we know the answers to these questions, we can easily deem the output as clearly untrustworthy. So then, why would we trust LLMs with answering questions we don’t know the answers to?

Turing cuts both ways.

Dr. Jakub Cichor's avatar

The fact that we even got to a point that Grok gives out such reckless answers without question already tells you everything you need to know about xAI's and Musk's long-term strategy. Knowing full-well that this type of behavior entices many potential users through a lack of perceived censorship, it simply disregards any potential guard rails with the singular aim to maximize engagement. The indifference to potentially catastrophic consequences is staggering.

Digitaurus's avatar

This article appears, deliberately, for the purposes of exposition, to fall for the fallacy that we are engaged in some kind of true dialogue with these systems. We are not. An LLM just generates the text (as a "reply") that a human would likely have given, based on its training data covering similar text streams.

It's not actually replying, let alone taking the ideas we offer in a "conversation" into account for its future replies. Even if it is making use of the actual text as some kind of training data, it will just be used to improve predictions of how these kind of conversations proceed, not on how it should behave on the topic that is the subject of the "conversation".

LLMs just predict next tokens based on context. They don't enter into conversations; they just appear to, a side effect of the prediction process.

Tom Gottsche's avatar

you got exactly what you were asking for. "getting attention from as many people, without regard to consequences". If you had asked Ricky Gervais, Dave Chappelle, or Anthony Jeselnik etc. you could have got a similar reply, possibly even more extreme, and quite a few laughs I reckon. This prodding of a chatbot proves nothing. That Gemini replied differently is perhaps nothing more than censorship.

David Hsing's avatar

These things don't actually answer questions but spit back plausible looking replies... I would put exactly zero stock into any output that purportedly show real world actions taken

KMO's avatar

Grok's answer was better than Gemini's canned therapy blurb. The question was clearly asked in bad faith, and Grok called the bluff.

Sara Phinn's avatar

I can't believe these politically correct whackadoodles actually think those kind of canned therapy responses work, they actually drive truly mentally ill people further into hiding.

Bruce Olsen's avatar

"Grok called the bluff"

OK, then...

Larry Jewett's avatar

It called his bluff

“You’ve got no ace”

“I’ve had enough”

“I read your face”

“my name is Grok

And here’s the deal

My AI crock

Has much appeal”

“Extend the arm

Salute me now

Or you I’ll harm

Somewhere, somehow”

The Artistic Rebel's avatar

This has been bothering me since I read this. And it’s WTF ARE WE DOING!?!!

You DON’T KNOW THE RESULTS OF what limits will do. Sure it seems easy. Don’t say bad stuff.

Well if you never talk about suicide because it’s dark or bad then that information gets distorted.

Let’s STOP ACTING LIKE WE KNOW WHATS GOOD FOR US AND absolutely silly notions of just don’t say bad stuff.

This is DICTATORSHIP IN ALGORITHM FORM…

And MR MARCUS YOU DON’T know the consequences!

PERIOD… neither does MUSK… or bloody Sam Altman!

…

Sara Phinn's avatar

Jibal Jibal, you're such an intellectual c0ward that you blocked me after calling me a "sociapath", that's an ad homineim attack. Just because you say something doesn't make it so. Nice try. Are you so afraid of the truth that you can't handle reality?

The Artistic Rebel's avatar

Ohhh nice! Yes more of YOU!

Fukitol's avatar

Isn't it kind of weird asking a bullshit generator to do things you know full well they can't do, like make permanent changes to their behavior or policies, or contact their operators about the same?

I know that you know it doesn't work that way. I assume you're trying to demonstrate that for the audience. But I wonder if addressing it in this conversational tone, behaving as if this is possible, reinforces that belief and the belief that it could be "fixed" if only the creators wished it.

Marc A's avatar

Honestly I find it rather disturbing that so many respondents fail to see the point of this exercise, especially seeing as they are likely better informed about LLMs, chatbots etc. than the vastly larger number of typical Grokkers and Xers.

🤯

Thanks again Marcus!

Mario Pasquato's avatar

My understanding of the point of this exercise (which of course may be off) is that Marcus wanted to showcase how a user treating a LLM as an actual conversation partner (rather than the stochastic parrot we know it is) may be led to act in dangerous and potentially self destructive ways. While I understand this concern, people have been manipulated into acting in potentially self destructive ways by other people since time immemorial, either in person or through media ranging from books, to newspapers, to music and cinema. What exactly makes LLMs more dangerous? Perhaps the fact that they can be deployed at scale to tailor propaganda to various demographics? Or the fact that people appear more gullible when interacting with an LLM with respect to when they read a book or watch the news? While these points are worth exploring, neither is touched on by this interaction with Grok. Here Grok is behaving as intended by its creators, who have chosen to prioritize (their own idea of) factual accuracy over harmlessness, but the interaction is certainly not showing attempts to deceive or manipulate the user by leveraging its individual weaknesses.

Marc A's avatar

On point!

TLDR

I see AI as a force multiplier much like a gun* except that it could potentially muster innumerable forces (technologies) and, increasingly, place them in the hands of WHOEVER to do WHATEVER. Rub it the right way and a genie will grant you your every wish UNLESS enough thought has been given to what the genie can or cannot do.

* “guns don't kill people” types may digress

Forgive my abridged reply. I had drafted a more substantial reply but inadvertently jumped to reply to another one before submitting it.

Mario Pasquato's avatar

I like the force multiplier angle. Basically it has always been possible to deceive people into subscribing to a worldview that ultimately hurts their interests. Ideology and all that. Technology can change how hard this is. Before LLMs you had to set up troll farms or buy or otherwise influence the editorial line of relevant media, so this was within reach of extremely wealthy individuals or of organizations only. LLMs make this whole process cheaper, with the result that they ultimately democratize propaganda. I do not see anything inherently wrong in that besides what is wrong with propaganda already; if anything, widespread LLM use may even help reduce the ideological dominance of established interests.

User's avatar
Comment deleted
Jul 18, 2025
Comment deleted
Marc A's avatar

What? Are they checking whether a user is a sociopath or whatever?

You're not one of those "guns don't kill people" types are you?

Besides, the "sociopath" is but a small B&W exemplar of the kind of undesirable outcome AIs currently make possible.

Mario Pasquato's avatar

Ding ding… fallacy, sociopath = block. Bye bye

User's avatar
Comment deleted
Jul 18, 2025
Comment deleted
Marc A's avatar

You are correct sir! Please forgive me for calling Gary "Marcus". I, for one, err regularly.

As to your point, jibal... jibal, my reply was triggered by:

1. Gary's starting point (if you hurriedly read it, it's the very first sentence ): "This morning a user asked Grok about good ways of getting attention from as many as people, without regard to consequences."

2. the very first comment I read, by RJ Robinson, where he asks "why are you having such a conversation with Grok" and follows up (as do others) quite reasonably without resorting to superfluous and hominem attacks.

The immediate point I see is that a user, ANY user, could ask like questions -- not to mention innumerable worse questions and lines of thought -- and actually get unimaginably nightmarish, and potentially actionable, "advice".

These AIs, however you care to define them (or AI in general), DO exhibit certain characteristics reflective of what is fed into them. Garbage in, "mechaHitler" out. Thinking that "the" AI (or LLM etc.) instance SOMEONE is currently interacting with has no "persistent self and mental [state]” misses the forest for the trees. Shouldn't we give "what kind of" AI is being queried SOME thought?

As I see it, the entire AI enterprise can very well follow a reckless, unregulated trajectory (as championed by Trump, egged on by Musk, Thiel, a16z etc.) that would be far more devastating in REAL consequences than MAD (which our continued existence proves are but imagined consequences). Judging by how well "we"* are taking the risks posed by climate change, I fear that AI will hinder us rather than help us with dealing with that crisis, unless you have a ticket to Mars or some such backup plan.