As somebody who works intensely with these AI tools, and knows their failure modes too, I will say that the value is huge. You may not care, if not in the industry, but I do not need a scientific study to prove to me I know what gives me value.
So, obviously It is possible to be confident and be wrong you may feel you are right and I am sure you can understand someone could be skeptical of your experience translating or being objectively true.
Also, It is possible to be confident and both right or wrong depending on context and perspective.
You could be right that you are more productive in the short term. But the question remains are you productive in the long term? Are there diminishing or negative returns on different timescales with this tool are you already there?
It is very easy to fool yourself by short term results and is quite hard to be objective about the long term. Hence why we need more science here.
Personally, based on my understanding of cognition, and how it relates to programming, I think the cognitive debt of overusing these tools is seriously underestimated by programmers not versed in cognitive psychology.
And if I'm wrong I ship less code than i otherwise could have. But with proper vetting of ideas that may actually be a boom as I don't waste time chasing every shiny object.
But if your wrong we dangerously degrade many systems as well as our own skills.
Again why I feel we need more science and less hype here.
With AI, I have a lot less patience for debugging than I used to. I can surely see getting rusty there. However, freed from that, I can spend more time thinking about the bigger issues. I am better at seeing the forest, and not the trees.
Long-term, life calibrates everybody. Bugs and failure teach you where to get better. And where to trust the bot less.
Debugging is the central skill of programmers though right? And knowing the systems due to having built them aids debugging.
And Sure systems level work can be quite satisfying, I can see that POV. That would be what I would focus on if I had to deal with these tools.
But knowing how reliable any given component is I feel needs some time down in the forest as well. Its also a great way to keep and maintain programming skills which is a different skillset than systems level knowledge and skills.
Anyway, I am not going to try to convince you to change your mind. I would like to get your honest answer on a few things though since you seem to be responding sincerly.
Curious how you are mentally handling the work.
Do you find your work more or less enjoyable? More or less taxing?
Do you still code manually on your own projects for fun? In other words did you enjoy coding for the intrinsic value before hand or was it just about producing features for you?
I love coding, in fact I do more of it than is healthy. As all programmers know, coding is like solving puzzles, sometimes it gets hold of you and one can't rest till something is figured out.
The annoying part of coding is looking up API, making changes in 10 files propagating a parameter, and wasting hours because one swapped by mistake two variables.
Coding now with AI is a lot more fun. A lot more. When tired I would not trust myself to code by hand as a screw up would waste precious hours next day backtracking.
The AI is able to keep remarkably well in mind the details. Way better than people. The AI screws up at the big picture, so that's your job. So it is team work. Even with less energy I can do more done if I have it handle some stuff while I keep it on track on occasion.
I am sure it does provide you with great value. It does for me to. Especially for research or smaller projects. But that doesn't mean that one should downplay or ignore the difficulties, dangers, and problems encountered using AI for code in production for large scale commercial projects, where it looks great, and then all of the sudden you can run into a significant problem that turns the productivity gain into productivity losses compared to not using AI.
I've been doing scientific and commercial software for 20 years. Also used CoPilot from early on, then switched to GeminiCLI, and now working with Claude Opus.
I assign to Claude work very carefully. For infrastructure and computer vision it is a great multiplier.
Why would you debug the code in 10 years time? Just vibe the new version from scratch, with new and updated features ... in 10 years time that will be a couple of hours work.
Here’s our big problem: where are the humans going to come from who are needed to fix the mess, when software firms with googly eyes for Claude stop training young engineers?
Cranky Frankie: As a boomer, I KNOW both the value of that, but also that there are probably as many boomer-clunkers as there are Gen z's and x's. And no one wants to slow down or speak with more clarity :o/
But it's a serious and centuries-old problem. There is nothing like having an historical context to understand what's going on now-today. However, knowing its potential significance always runs right into the question: How do you get that across to those who don't have that context yet?
AI "Natives" will oust the old Engineers for money reasons and then when shit hit the fan whoever is left that predated AI are going to be crushed in work.
This is exactly the right question. You could say the same thing about so many fields. Who is going to make the big advances in knowledge if we all rely on AI.
As a SWE, the problem is that the more AI agents write code, the more developers become less competent. This is a serious issue because, no matter how much they improve, LLMs will always remain somewhat unreliable. And their unreliability is becoming increasingly harder to detect, because everything seems to work fine.
AI coding tools are useful but should be used with common sense, and as deadlines become tighter "because there is AI", teams will increasingly end up doing basically vibecoding, as it's already happening.
I think this is a further ramification of the "automation/autopilot" issue. The human user gets used to the AI producing apparently good/working code most of the time. As a result, the sheer volume of code produced results in relatively little inspection for obvious bugs. The bugs slip through, there is insufficient documentation, and so the errors bite you down the road. As no one has written the code, it is harder to fix, and your expertise to do so has left more interesting work.
As Cory Doctorow has written, while teh suits like more code written for less cost, the code is just more places for attacks. The best code is teh code you don't have to write. It won't just be the in-house code, but the code libraries and dependencies the in-house code requires that may be buggy or vulnerable to attacks.
Alex: There is no accounting for the human penchant for being bored (a contradiction in terms, I think). I used to find that RE-reading a book was filled with happy insights. Now I wonder who even reads books at all. My guess the problem shares some deep psychological distortion with being overweight and gambling.
I have a PhD in AI and worked in commercial software engineering for over 30 years. I was an ardent fan of agile and automated testing since it's inception. Hired hundreds of developers over the years from new grad through to Principal level.
When I started reading that companies were hiring fewer new grad and junior developers and "replacing" them with GenAI it took me 2 nanoseconds to spot the idiocy of this approach: What does the company do a few years (or even months!) down the road when the code needs debugging, enhancing - and, generally, maintaining? How do you fix the issues the GenAI can't fix when you've got this staffing "hole"? A hole that comprises a deficit in basic software engineering skills as well as domain knowledge of the company's core business.
Expensive deep dives with senior developers will work. At at cost. But you'll need to repeat this again and again. The bigger the code base, the more numerous and more complex (and more expensive and more risky) the debugging of it (typically). With Comp Sci intake numbers dropping throughout most universities there well be fewer viable candidates to bring on board as new or junior developers. The same developers, it should be said, who would have turned into your indispensible, kick-ass senior developers of the future! I'm sure the pro-GenAI CTOs got their inflated bonuses for cutting HR costs in the past couple of years, but they've solved nothing. They've just cashed their cheques and kicked the can of worms slightly further down the road. Probably for someone else to deal with.
GenAI to assist (and indeed to teach and improve) knowledgable developers, yes. GenAI to cut costs on hiring your future essential developers. Dumb as a bag of bricks.
Or, find the sweet spot where AI augments you and allows you to spend more time on higher level work and on reviews. It is foolish to refuse any help. But one has to do a thorough job.
Having been a developer for over 50 years, I could never get coders to verify their own code. And companies would never pay for proper testing or proper functional design invariably resulting in design flaws and errors in boundary conditions, error handling paths, and pointers. Coders are notoriously poor at testing code. They don't have the patience or the skill set. You need dedicated testers. Automated testing finds the obvious buys but has never been sufficient. Where AI coding requires more testing than humans, not less, companies will never pay for the testers.
Yes. They've been using consumers as guinea pigs for decades. In addition, I suspect as in proofreading, it is never wise to proof your own work: too much bias NOT to find errors or assume too much and miss mistakes.
Actually, I've worked with some excellent coders. A great coder can outperform a regular coder by 5:1 but they usually only cost 50% more so I focus on the best coders I can find. But they think in terms of what the user is going to do right, not in terms of what could go wrong. And they assume the error paths they have coded work with a few automated test cases. Of course, those are some pretty generalized statements that I made now and originally. I'm good at that. The nature of the bugs depends a lot on the language. In the days of C, It was a disaster. These days, errors come more from data that the programmer didn't consider and design issues where they didn't follow the requirements or the spec or there is no spec or the spec sucks because the designer didn't consider all the things a user might need or try. And there is a lot of agile programming where they are prototyping so they are thinking that it's not the final code, but it always seems to be the final code so the bugs remain. Automated testing is good but is only as good as the test cases defined which are typically inadequate. I believe that the MVP is really the minimum useful product without bugs from a user's perspective. 50 years ago, people didn't understand software development or requirements analysis. They would give us a rough description of what they wanted, we would go off into a closet and code for 2 or 3 months, and then give them the program. Vibe coding and a lot of coding these days are the same. It's just an excuse to not spend the time figuring out what the user actually needs for their process, what the results should look like, and the optimal design for the system. Quick and dirty is the rule and rework and debugging is never considered. It's just like 50 years ago but you don't have to sit in a closet. It was the same at the beginning of HTML. We seem to discover the importance of defining the problem, a voice of the customer up front, and test cases to define success over and over. This isn't the coder's fault unless they are also the designer and business manager.
Come on, really? This is a natural cognitive bias. Designing good tests for code you've written yourself is really hard. It is smart to have dedicated testers.
And code verification is the least fun part of the job, so of course it's difficult to get people to do it.
So, in essence, testing one's own code is difficult therefore it's impossible that some actually do it.
I conclude that you've never worked with people who do. I'm telling you that they most definitely exist. They are called professionals and take pride in doing the hard part well.
I realize this isn't common in large organizations. Then again, the overall quality of software they produce doesn't really recommend their practices.
Of the developers I know that use AI the point is to vibecode and rarely check on anything. Ship faster above all things. Just hopped back into 2 codebases where core dev is AI generated, tests don't even pass. In my ideal world humans would pair program with the AI reviewing as it went and a human would write meaningful tests (and obviously run them) and it's still peer reviewed by another human; but then code will be "expensive" again and that's not the goal of the industry now.
All the best practices we've learned over the decades take a back seat to "fast fast fast"; so no one believed in them anyway I suppose, too bad cause they worked.
Peter: Even a brief look at some other cultures will show that not everyone is like that. Some identify more with tradition and the past than with always fast-forwarding. Though differences can always be found, as a cultural movement, "fast fast fast" is a particularly American trait.
I worked at a FAANG for 18 years. Classic phenomenon: junior eng seeling promotion created a.myriad Artifacts, demonstrates improvement in key Metrics, gets Promoted, and promptly Disappears to another team, leaving someone else to deal with their mess.
After reading all the notes up to now, I think FULL DISCLOSURE of AI usage in anything published is essential and, if the rule is ignored twice, punishable by episodes of computer-burnings and years in prison, if not worse where people die for one's carelessness or deliberate ignorance.
I have to wonder if there are ANY protocols for checking these things, and for holding the person(s) responsible for setting up the situation, then pushing the buttons accountable. If there were stiff penalties, and protocols in place, there would be better oversight. At least that.
It all comes back to that ML, LLMs and associated are not in fact "AI" in any meaningful sense. I just have no idea why these mega corps would pile so much money in to dead end technology. It's utterly baffling.
Its because they (CEOs) are taken in by the hype (generated largely by vested interests), for the simple reason that they don't understand what we currently call "AI" is really all about (i.e., strengths, weaknesses).
AI coding tools should be fine in the right hands (e.g. Oleg in this thread) but they are so easily misused out of ignorance and "overtrust". Software developers have complained for decades that their line managers don't actually know that much about software (back to the "fast, faster" argument) but AI is set to make this situation far worse.
Not a programmer here, but seems to me another reason they're misused is laziness. Thanks to social media and ubiquitous smart phones, research shows people lose their attention span. I know intelligent people who can no longer concentrate long enough to read a book and who circulate fake AI "news" without taking a few seconds to first verify it. We need to slow down and think deeply, not impulsively move fast and break things for instant profit.
Gary: I hope this is okay to post the below here as a sideline. In my inbox just now is "breaking news" from the New York Times where the headlines speak for themselves about what our dear leader is up to in the incremental take-down of an otherwise working democracy:
FIRST HEADLINE: Unlike past U.S. conflicts, the Iran attacks are opposed by most Americans.
SECOND HEADLINE: The Trump administration and Penn are meeting in court after the government demanded a list of Jewish students and staff.
Don't worry ....
'It works for me and it's only going to get better!'
Because anecdotal evidence and unsupported assertion is the bedrock upon which Science and Technology is built.
As somebody who works intensely with these AI tools, and knows their failure modes too, I will say that the value is huge. You may not care, if not in the industry, but I do not need a scientific study to prove to me I know what gives me value.
So, obviously It is possible to be confident and be wrong you may feel you are right and I am sure you can understand someone could be skeptical of your experience translating or being objectively true.
Also, It is possible to be confident and both right or wrong depending on context and perspective.
You could be right that you are more productive in the short term. But the question remains are you productive in the long term? Are there diminishing or negative returns on different timescales with this tool are you already there?
It is very easy to fool yourself by short term results and is quite hard to be objective about the long term. Hence why we need more science here.
Personally, based on my understanding of cognition, and how it relates to programming, I think the cognitive debt of overusing these tools is seriously underestimated by programmers not versed in cognitive psychology.
And if I'm wrong I ship less code than i otherwise could have. But with proper vetting of ideas that may actually be a boom as I don't waste time chasing every shiny object.
But if your wrong we dangerously degrade many systems as well as our own skills.
Again why I feel we need more science and less hype here.
With AI, I have a lot less patience for debugging than I used to. I can surely see getting rusty there. However, freed from that, I can spend more time thinking about the bigger issues. I am better at seeing the forest, and not the trees.
Long-term, life calibrates everybody. Bugs and failure teach you where to get better. And where to trust the bot less.
Debugging is the central skill of programmers though right? And knowing the systems due to having built them aids debugging.
And Sure systems level work can be quite satisfying, I can see that POV. That would be what I would focus on if I had to deal with these tools.
But knowing how reliable any given component is I feel needs some time down in the forest as well. Its also a great way to keep and maintain programming skills which is a different skillset than systems level knowledge and skills.
Anyway, I am not going to try to convince you to change your mind. I would like to get your honest answer on a few things though since you seem to be responding sincerly.
Curious how you are mentally handling the work.
Do you find your work more or less enjoyable? More or less taxing?
Do you still code manually on your own projects for fun? In other words did you enjoy coding for the intrinsic value before hand or was it just about producing features for you?
I love coding, in fact I do more of it than is healthy. As all programmers know, coding is like solving puzzles, sometimes it gets hold of you and one can't rest till something is figured out.
The annoying part of coding is looking up API, making changes in 10 files propagating a parameter, and wasting hours because one swapped by mistake two variables.
Coding now with AI is a lot more fun. A lot more. When tired I would not trust myself to code by hand as a screw up would waste precious hours next day backtracking.
The AI is able to keep remarkably well in mind the details. Way better than people. The AI screws up at the big picture, so that's your job. So it is team work. Even with less energy I can do more done if I have it handle some stuff while I keep it on track on occasion.
I am sure it does provide you with great value. It does for me to. Especially for research or smaller projects. But that doesn't mean that one should downplay or ignore the difficulties, dangers, and problems encountered using AI for code in production for large scale commercial projects, where it looks great, and then all of the sudden you can run into a significant problem that turns the productivity gain into productivity losses compared to not using AI.
I do work with these things daily, and know their limits too. How about you? Any direct experience?
I've been doing scientific and commercial software for 20 years. Also used CoPilot from early on, then switched to GeminiCLI, and now working with Claude Opus.
I assign to Claude work very carefully. For infrastructure and computer vision it is a great multiplier.
🤣🤣🤣💯
Why would you debug the code in 10 years time? Just vibe the new version from scratch, with new and updated features ... in 10 years time that will be a couple of hours work.
Right?
Right?
ha ha ha ...
No vibe. Honest work with AI results in great results. If one is incompetent and vibes, one is doomed.
Here’s our big problem: where are the humans going to come from who are needed to fix the mess, when software firms with googly eyes for Claude stop training young engineers?
Boomers called back from retirement at great expense might help with understanding.
COBOL 2.0 ...
Boomers? Those are the managers forcing AI adoption or holdovers doing COBOL in a corner in a bank.
Who is going to be fucked is the generation X.
... again
Cranky Frankie: As a boomer, I KNOW both the value of that, but also that there are probably as many boomer-clunkers as there are Gen z's and x's. And no one wants to slow down or speak with more clarity :o/
But it's a serious and centuries-old problem. There is nothing like having an historical context to understand what's going on now-today. However, knowing its potential significance always runs right into the question: How do you get that across to those who don't have that context yet?
AI "Natives" will oust the old Engineers for money reasons and then when shit hit the fan whoever is left that predated AI are going to be crushed in work.
Ought to be interesting when the Unix bomb goes off in 2038.
Good point. It’s dependent on us ultimately.
This is exactly the right question. You could say the same thing about so many fields. Who is going to make the big advances in knowledge if we all rely on AI.
And how is AI going to advance if we don't have any new advances in knowledge to train it on?
"Open the pod bay doors." -- overheard at Amazon engineering meeting
As a SWE, the problem is that the more AI agents write code, the more developers become less competent. This is a serious issue because, no matter how much they improve, LLMs will always remain somewhat unreliable. And their unreliability is becoming increasingly harder to detect, because everything seems to work fine.
AI coding tools are useful but should be used with common sense, and as deadlines become tighter "because there is AI", teams will increasingly end up doing basically vibecoding, as it's already happening.
I think this is a further ramification of the "automation/autopilot" issue. The human user gets used to the AI producing apparently good/working code most of the time. As a result, the sheer volume of code produced results in relatively little inspection for obvious bugs. The bugs slip through, there is insufficient documentation, and so the errors bite you down the road. As no one has written the code, it is harder to fix, and your expertise to do so has left more interesting work.
As Cory Doctorow has written, while teh suits like more code written for less cost, the code is just more places for attacks. The best code is teh code you don't have to write. It won't just be the in-house code, but the code libraries and dependencies the in-house code requires that may be buggy or vulnerable to attacks.
"Code is a liability (not an asset)" - https://pluralistic.net/2026/01/06/1000x-liability/
Alex: There is no accounting for the human penchant for being bored (a contradiction in terms, I think). I used to find that RE-reading a book was filled with happy insights. Now I wonder who even reads books at all. My guess the problem shares some deep psychological distortion with being overweight and gambling.
I have a PhD in AI and worked in commercial software engineering for over 30 years. I was an ardent fan of agile and automated testing since it's inception. Hired hundreds of developers over the years from new grad through to Principal level.
When I started reading that companies were hiring fewer new grad and junior developers and "replacing" them with GenAI it took me 2 nanoseconds to spot the idiocy of this approach: What does the company do a few years (or even months!) down the road when the code needs debugging, enhancing - and, generally, maintaining? How do you fix the issues the GenAI can't fix when you've got this staffing "hole"? A hole that comprises a deficit in basic software engineering skills as well as domain knowledge of the company's core business.
Expensive deep dives with senior developers will work. At at cost. But you'll need to repeat this again and again. The bigger the code base, the more numerous and more complex (and more expensive and more risky) the debugging of it (typically). With Comp Sci intake numbers dropping throughout most universities there well be fewer viable candidates to bring on board as new or junior developers. The same developers, it should be said, who would have turned into your indispensible, kick-ass senior developers of the future! I'm sure the pro-GenAI CTOs got their inflated bonuses for cutting HR costs in the past couple of years, but they've solved nothing. They've just cashed their cheques and kicked the can of worms slightly further down the road. Probably for someone else to deal with.
GenAI to assist (and indeed to teach and improve) knowledgable developers, yes. GenAI to cut costs on hiring your future essential developers. Dumb as a bag of bricks.
It's a death spiral.
It's going to take years to play out.
It'll be a giant effing mess.
I can't think of many things more soul-crushing than reducing my career to code auditing.
I think I'm going to camp in the woods.
The AI is blamed but the problem is humans not verifying outputs. AI is a tool. Use it properly and understand where it succeeds and where it fails.
It's a human and a workflow problem for businesses.
i actually wrote about this yesterday in the essay on Dario and checking outputs
Or don't use it at all. Keep growing your development skills and refuse to surrender them in the hype-filled melee.
Excellent, human developers who didn't give in to FOMO will be very needed for a long time to come.
Or, find the sweet spot where AI augments you and allows you to spend more time on higher level work and on reviews. It is foolish to refuse any help. But one has to do a thorough job.
no true Scotsman, eh?
Having been a developer for over 50 years, I could never get coders to verify their own code. And companies would never pay for proper testing or proper functional design invariably resulting in design flaws and errors in boundary conditions, error handling paths, and pointers. Coders are notoriously poor at testing code. They don't have the patience or the skill set. You need dedicated testers. Automated testing finds the obvious buys but has never been sufficient. Where AI coding requires more testing than humans, not less, companies will never pay for the testers.
Yes. They've been using consumers as guinea pigs for decades. In addition, I suspect as in proofreading, it is never wise to proof your own work: too much bias NOT to find errors or assume too much and miss mistakes.
100%
That means you've worked with bad coders for 50 years. Sorry..
Actually, I've worked with some excellent coders. A great coder can outperform a regular coder by 5:1 but they usually only cost 50% more so I focus on the best coders I can find. But they think in terms of what the user is going to do right, not in terms of what could go wrong. And they assume the error paths they have coded work with a few automated test cases. Of course, those are some pretty generalized statements that I made now and originally. I'm good at that. The nature of the bugs depends a lot on the language. In the days of C, It was a disaster. These days, errors come more from data that the programmer didn't consider and design issues where they didn't follow the requirements or the spec or there is no spec or the spec sucks because the designer didn't consider all the things a user might need or try. And there is a lot of agile programming where they are prototyping so they are thinking that it's not the final code, but it always seems to be the final code so the bugs remain. Automated testing is good but is only as good as the test cases defined which are typically inadequate. I believe that the MVP is really the minimum useful product without bugs from a user's perspective. 50 years ago, people didn't understand software development or requirements analysis. They would give us a rough description of what they wanted, we would go off into a closet and code for 2 or 3 months, and then give them the program. Vibe coding and a lot of coding these days are the same. It's just an excuse to not spend the time figuring out what the user actually needs for their process, what the results should look like, and the optimal design for the system. Quick and dirty is the rule and rework and debugging is never considered. It's just like 50 years ago but you don't have to sit in a closet. It was the same at the beginning of HTML. We seem to discover the importance of defining the problem, a voice of the customer up front, and test cases to define success over and over. This isn't the coder's fault unless they are also the designer and business manager.
Come on, really? This is a natural cognitive bias. Designing good tests for code you've written yourself is really hard. It is smart to have dedicated testers.
And code verification is the least fun part of the job, so of course it's difficult to get people to do it.
Well said. I don't know if you're a good coder but you're a good writer. You reduced everything I said two four sentences. Good job!
So, in essence, testing one's own code is difficult therefore it's impossible that some actually do it.
I conclude that you've never worked with people who do. I'm telling you that they most definitely exist. They are called professionals and take pride in doing the hard part well.
I realize this isn't common in large organizations. Then again, the overall quality of software they produce doesn't really recommend their practices.
Of the developers I know that use AI the point is to vibecode and rarely check on anything. Ship faster above all things. Just hopped back into 2 codebases where core dev is AI generated, tests don't even pass. In my ideal world humans would pair program with the AI reviewing as it went and a human would write meaningful tests (and obviously run them) and it's still peer reviewed by another human; but then code will be "expensive" again and that's not the goal of the industry now.
All the best practices we've learned over the decades take a back seat to "fast fast fast"; so no one believed in them anyway I suppose, too bad cause they worked.
Peter: Even a brief look at some other cultures will show that not everyone is like that. Some identify more with tradition and the past than with always fast-forwarding. Though differences can always be found, as a cultural movement, "fast fast fast" is a particularly American trait.
When that IoT garage door opener gives my bank pin to thieves, I'll be suing. It's a whole new growth area for the plaintiff's bar.
So hold programmers who use AI accountable for errors made? Then you'd have to make using AI optional., which might be smart.
. . . or do whatever a business can to provide incentives? They are not stupid--they can find them?
. . . or maybe they ARE stupid.
"But wait, I never asked you to touch that part, that part was correct and working!"
Every time.
Infuriating.
I worked at a FAANG for 18 years. Classic phenomenon: junior eng seeling promotion created a.myriad Artifacts, demonstrates improvement in key Metrics, gets Promoted, and promptly Disappears to another team, leaving someone else to deal with their mess.
And now we have a machine to automate this.
(no edit button huh?)
Looks like Cory Doctorow was on to something when he said “code is a liability “.
After reading all the notes up to now, I think FULL DISCLOSURE of AI usage in anything published is essential and, if the rule is ignored twice, punishable by episodes of computer-burnings and years in prison, if not worse where people die for one's carelessness or deliberate ignorance.
I have to wonder if there are ANY protocols for checking these things, and for holding the person(s) responsible for setting up the situation, then pushing the buttons accountable. If there were stiff penalties, and protocols in place, there would be better oversight. At least that.
SchAIdenfreude.
It all comes back to that ML, LLMs and associated are not in fact "AI" in any meaningful sense. I just have no idea why these mega corps would pile so much money in to dead end technology. It's utterly baffling.
Its because they (CEOs) are taken in by the hype (generated largely by vested interests), for the simple reason that they don't understand what we currently call "AI" is really all about (i.e., strengths, weaknesses).
AI coding tools should be fine in the right hands (e.g. Oleg in this thread) but they are so easily misused out of ignorance and "overtrust". Software developers have complained for decades that their line managers don't actually know that much about software (back to the "fast, faster" argument) but AI is set to make this situation far worse.
Not a programmer here, but seems to me another reason they're misused is laziness. Thanks to social media and ubiquitous smart phones, research shows people lose their attention span. I know intelligent people who can no longer concentrate long enough to read a book and who circulate fake AI "news" without taking a few seconds to first verify it. We need to slow down and think deeply, not impulsively move fast and break things for instant profit.
It should have been obvious from the start for two reasons:
1) Humans write crap code. AI is trained on human code. AI writes crap code (especially when it comes to secure code, which humans suck at.)
2) AI are probabilistic (so are humans), not deterministic. Which means a (possibly very) significant part of the time, they will generate crap.
I like the comment that said "Oh, well, it works for me and it will get better" - said no engineer ever (hopefully).
Gary: I hope this is okay to post the below here as a sideline. In my inbox just now is "breaking news" from the New York Times where the headlines speak for themselves about what our dear leader is up to in the incremental take-down of an otherwise working democracy:
FIRST HEADLINE: Unlike past U.S. conflicts, the Iran attacks are opposed by most Americans.
SECOND HEADLINE: The Trump administration and Penn are meeting in court after the government demanded a list of Jewish students and staff.