On hype, and the unbearable banality of ChatGPT’s poetry
A new AI study is making the rounds, claiming that ChatGPT can write poetry that is “indistinguishable” from William Shakespeare.
Um, by whom?
§
My frequent collaborator Ernest Davis had a careful look at the study, methods, materials, etc, and not just the headline.
His dissection of the poetry study makes for an entertaining read. I don’t want to spoil all the fun, but I’ll say this:
Come for quotes like this “the AI poems seem like imitations that might have been produced by a supremely untalented poet who had never read any of the poems he was tasked with imitating, but had read a one-sentence summary of what they were like”.
Stay for the Appendix, entitled “Particularly terrible lines”.
§
As Ben Riley opined on my new favorite social media site, Bluesky (you can find me there at @garymarcus.bsky.social), it’s a great takedown:
Davis’s full critique (and a link to the study itself) can be found here.
Gary Marcus hopes he has taught you by now to never trust the hype.
Unfortunately LLMs are trying to confound what people mean by creativity. These authors trained the model on the entire bodies of work created by those poets in the first place. And as you can see here, even then the outputs are just mundane mimicry.
Poets are seldom so derivative, let alone regurgitating "accessible" language in someone else's style!
Why are the inventors so obsessed with replacing himan creativity? It is our souls.