Discussion about this post

User's avatar
rif a saurous's avatar

Do you consider works like "Memorizing Transformers" (https://arxiv.org/abs/2203.08913), which augments transformers with an explicit external memory allowing much longer contexts, to already represent a "paradigm shift?"

Expand full comment
DABM's avatar

'Rising star Abebe Birhane has written a withering criticism of the ways in which LLMs rely of objectionable data scraped from the internet. '

This, however important in itself, is basically irrelevant to whether 'scaling will get us to AGI' no? You could have a very intelligent AI that had racist and sexist biases, just like some distinguished scientists have also been racist cranks. If scaling got us that it would be *bad*, but it would still mean Alexander was more right about the potential of just scaling stuff up than you are.

Trying to make debates on fairly specific claims about AI, like the one about whether scaling will lead to human-level-or-better performance on all or most tasks, really about whether you are generically pro/anti AI (or worse still tech as industry), strikes me as a recipe for confusion and polarization.

Expand full comment
25 more comments...

No posts