Most people don’t think GenAI has improved their productivity
Damning new study from Upwork
Wow! Incredibly damning new study on GenAI from the Upwork Research Institute, via Baldur Bjarnason (whose wonderful essay on LLMs and how they fool the gullible I discussed in April).
Bjarnason‘s new essay, The other shoe dropping on ‘AI’ and office work, reports that Upwork found that
“Nearly half (47%) of workers using AI say they have no idea how to achieve the productivity gains their employers expect. Over three in four (77%) say AI tools have decreased their productivity and added to their workload in at least one way.”
Says Bjarnason, “It’s quite unusual for a study like this on a new office tool, roughly two years after that tool—ChatGPT—exploded into people’s workplaces, to return such a resoundingly negative sentiment.”
§
Many coders and tech aficionados may love ChatGPT for work, but much of the outside world feels quite differently.
As Bjarnason points out, “it fits with the studies on the actual functionality of said tool: the incredibly common and hard to fix errors, the biases, the general low quality of the output, and the often stated expectation from management that it’s a magic fix for the organisational catastrophe that is the mass layoff fad.”
Read Bjarnason’s new essay here.
PS Microsoft Bing completely flubbed, 4/4, “people going yuck towards Generative AI”. Plus ça change, plus c'est la même chose.
We have been experimenting with GitHub CoPilot for a few months now and it has really only amounted to a fancier autocomplete. Often the same prompts generate different results or behavior, so between modifying the prompts and then correcting the code (e.g., unit tests), the productivity gains are minor at best.
I work for a Fortune 500 in a finance-related role and there's zero adoption of AI that I've seen. I've talked to IT folks involved and there's just now some investigation behind the scenes. Stuff like scanning pdfs of contracts for certain language seems on the table, but nothing has been implemented yet. I have to imagine plenty of other companies are taking this kind of wait and see approach.