Spinal Tap meets John Malkovich

What’s so funny about the need for world models?

As anyone of my generation would know, John Malkovich is famous for working on the 7.5th floor, in Spike Jonze and Charlie Kaufman’s surreal Being John Malkovich:

The movie This is Spinal Tap is famous, of course, for introducing the phrase “turn it up to 11”. If you don’t know the scene, take a moment to watch:

In celebration of the soon-to-be-released sequel of This is Spinal Tap, Spinal Tap II: The End Continues, my good friend Harry Shearer (below) who plays the band’s bass player, Derek Smalls …

The band, 41 years later, Derek Smalls (played by Harry Shearer) in the middle

…has been sending me a steady stream of generative AI fails.

Like this:

And this, um, unusual configuration of human and musical instrument:

I was particularly taken by one Harry sent this morning, an attempt to elicit an image of his character riding a unicycle on a beach. Especially when I studied it carefully.

Harry laughed at the “realistic waves”. Neither of us spoke of the fact that the image didn’t look a thing like him.

But it was the physics of the unicycle (no pedals, no spokes) and unicyclist that really got me. What is holding this guy up?

These systems can draw things that kinda look like the world, but they obviously don’t know what’s what or how things work outside of their training data. (Despite a lot of training data).

§

Me being me, I posted the pedal-less unicycle on X, whining about how it showed how bad the “world models” in these things could sometimes be:

But was the laugh on me?

Twitter being Twitter, somebody almost immediately scolded me. Bad model. Skill issue! You just need a better model! They snapped back with this:

And at first I was stunned by the refutation, produced by ChatGPT-5 Thinking. I can’t lie. This one was infinitely better than the first. The image was sharp! The wheel had spokes! Pedals, even! No model could have done this in 2022. The graphics are vastly better than they used to be. And this model was clearly better than the one I had ridiculed.

§

But then (once again) I looked closely.

For present purposes I will ignore the fact that the unicycle seems to have an reflection in the sand perpendicular to its shadow (leaving that to lighting experts), and ignore the fact that the unicycle seems maybe (not clear) not to have a seat, potentially posing certain painful problems for its rider. (I should know, having ridden a unicycle off and on for the last few decades. It’s the getting on seatless wheels that has me concerned.)

Instead, I ask you to focus on the bass.

Let’s zoom in, in fact:

2.5 tuners? and, wait for it, 4.5 strings!

Malkovich meets Tap! (Anyone remember this crossover between Mozart and Bach?)

Five strings near the head stock, four at the bridge. As one of Harry’s co-stars in A Mighty Wind, might say

Better graphics doesn’t necessarily entail a better understanding of the world. Of course you if run these systems many times, sometimes they will be right, sometimes they will be wrong — and you never know what you are going to get. Because they live in the land of correlations between images and text, not in the land of ideas.

§

For good measure, sound up, here’s a video version, courtesy Grok. Watch the tuners come and go! So much for object permanence.

And, as Harry noted to me in an email when I passed the video along, “the soundtrack has [Derek Smalls] playing guitar! It’s amazing how much a PhD level machine can get wrong!”

True fans read all the way to the end…

P.S. Coming September 12! Elton John! Paul McCartney! Questlove!

P.S.

Count the notches