Both LLMs and the AI labs that deploy them are misaligned with humans. What could possibly go wrong...? Luckily, LLMs are far less intelligent than most people perceive them to be. Even so...
Yes, the real current "x-risk" is the loss of our privacy. It is the theft of all our data.
There is no intelligence in the machine, but it is an incredible spy device and heuristic observer of human behaviors. The perfect tool for dystopian social engineers.
Related thoughts I had written previously ...
"The value of privacy can not be overstated. It is the foundation of both who we are individually and the societal order that makes up a free civilization. At threat is precisely our identity and the type of society that values and provides for freedom and liberty."
I don't understand why you all are missing the most important point: AGI is of least interest, because it is a crooked concept because it defines equal level in everything. How stupid. We have ALREADY and will have probably for years something else: AI is superior in increasing no. of tasks, but it will be inferior in a lot of others. No reason for relief "that AGI isn't reached!!!!" It started years ago, but the pace is incredible. As long as AI has been only superior in playing chess or Go, nobody really took notice. In 2025 AI is superior in answering nearly every question compared to even PhDs. The programming skills are not superior to humans by any measure, but in a lot of areas they are already, and for most projects of manageable size, AI will beat probably any human in 2025- at least what Open AI says, but even being comparable to best human programmers would be enough- and we have reached that point already ! In 2025 AI has a reached a point where it can replace "an increasing amount" of human paid work ! It doesn't matter, if it is for 1% of jobs only 1% of their work. The main point is:
There is no hard AGI line ! It is a jagged line! There will probably never be that hard line. It is HERE, it can do work that MATTERS form business perspective. THE LINE has been passed. You need a catchphrase: Call it ADI - "Artificial Dangerous Intelligence". This level is defined as "F*ck the AGI definition, if AI can perform already enough things better than humans that you should act as AGI is reached !"
Try looking at some of the "AI fails" and similar subreddits if you think AI is superior in answering nearly every question. Or, hey, just take a look at companies' own research on metrics like SimpleQA and PersonQA, hovering around 50% to 80% (so, 20% to 50% wrong).
This is something completely different. Show me a human who makes no errors. Where is the human, whose answer you can trust without further analysis what others are saying?? There is no. AI is not flawless and far from 100% correct, nobody with a clear mind would claim, every answer is correct; like every technology it has weaknesses. This is a big misunderstanding, that every mistake you find is a prove that AI is dumb. It is not.
That is just the special thing: Traditional computers have other weaknesses than humans, current AI has similar weaknesses. It doesn't know what it doesn't know.
Moreover for fails it is very important to say model and date, when it occured. Most showcases of errors are hard to repeat.
The thing is: > 90% fails are quite easy to avoid or easy to detect. E.g. Just asking the same model a question to verify it's answer, find very much problems. Combining multiple models and multiple prompts is a nother. We have just yet starting with a good "justify" step after answering. This will improve things strongly. E.g. Perplexity does a good job of avoiding most hallucinations with internet prove.
In each case: Mixing the failures of single models or finding weak points does not contradict the fact about there problem solving capabilities compared to humans which also fail.
This is like taking car failures as a proof that cars are a non-working technology.
You want to claim that they are superior at answering almost all questions, then the burden of proof is on your to provide evidence to contradict the data sources that I have provided, not write vague platitudes.
LLMs are as good or better than average humans at INTERPOLATION, but average humans exhibit little actual thought. They mostly believe they already know the answers, or have a "trusted source" for them - that is, somebody knows the answer and published it accessibly.
True human intelligence leverages rules verified by cross sensory knowledge or at least multiple pathways to produce accurate extrapolations. Books and text contain a large proportion of false information including much "common knowledge" which is culturally axiomatic but does not withstand falsification analysis.
This is why I seek texts written by the hands-on researchers rather than second hand summarirs or even supervisors who claim first authorship due to political hierarchies. Pit AI against the .1% who make breakthroughs in human knowledge and it fails.
Coming back to Jonah, finally I will let a reference here for the current progress and first signs of superiority.
"On the GPQA Diamond benchmark, o3 achieved an impressive 87.7%, well above the performance of human experts, reinforcing its strengths in problem-solving."
"On Codeforces, a competitive programming platform, o3 achieved an Elo rating of 2727, surpassing OpenAI's Chief Scientist's score of 2665."
Even if the results would be a bit "made-up" due to not fair training. The trend is what is important, not the numbers and the exact time and ratio of being superior in many questions to average PhDs !
I'm sorry. I feel like this is somewhat credulous. You have a lot of faith in the rigor of the testing procedures of these profit-motivated companies that have shown severe ethical lapses in the past, which is not warranted.
What is the GPQA Diamond benchmark? It is something not open to the public. What is the training data for OpenAI’s models or those of other companies? We don't know. Are the companies being honest by keeping the test questions (or very similar ones) out of the training set? We don't know. How much time and resources are they using to make sure a model gets the right answer? How do they determine whether the model gets the right answer? We. Don't. Know.
Anything that the companies release should be treated as an upper bound for performance, not as a scientific representation.
Similarly with the coding competition. Is the data contaminated with common questions? We don't know. We also know that Codeforces ratings go up to 4000, so that OpenAI employee, with all respect, is good, but not that good (and having a PhD is far less important than having practiced Codeforces problems a lot, by the by).
What we do see is information coming from everyday use where even the most models make easy mistakes that can be identified by users even with little expertise in the area, and metrics from the companies themselves which, taken as an upper bound, suggest the same thing.
How can we know that LLMs are "far less intelligent than most people perceive them"? Any machine smart enough to pass the Turing Test is smart enough not to, circa 1965.
Because genuine "machine intelligence" (things like continuous learning, continuous planning, constructing/maintaining internal world models (theories of the universe) sufficient for reliable prediction, understanding, and reasoning (e.g. induction, deduction, and abduction)) occur *inside* the machine. External observations such as evals, "Turing Tests" etc only measure externally-observable behaviour. It is not possible to reliably infer internal behaviour (i.e. genuine intelligence) from externally observed behaviour (inside, it could be just a massive lookup table). This is basic Chinese Room stuff. The only way to determine if a machine has genuine *intrinsic* intelligence is to look inside it (i.e. inside the NN), e.g. via mechanistic interpretability (MI). The current MI evidence seems to indicate that LLMs maintain very shallow world models and perform very weak reasoning (primarily analogical). So they seem to fall somewhere in between simple database lookup + autocomplete and genuine cognition. Mostly, the "intelligence" that you / most people perceive is primarily a consequence of (a) massive scale, and (b) anthropomorphisation, i.e. LLMs, as I said, are much less intelligent than people perceive them to be. An LLM's "intelligence" is more akin to a "stochastic mirage" than the real thing. Read this (AI-generated) paper: https://www.linkedin.com/posts/aaron-michael-turner_llms-through-the-lens-of-mechanistic-interpretability-activity-7340320787138011137-yHhH.
Aaron, you are right in one way. The answer is even easier: LLMs can be widely compared to a "system 1" intelligence of humans, the subconscious system. They are (obviously, I think) not made for "reasoning" as our "system 2" (see Kahnemann). As you write the "world models are quite shallow". I do not disagree, but I would put, given O3/O4, deep reasoning, etc. You get not a system with "real" intelligence, like a human, but you get something else capable of astounding tasks.
Your are wrong in another way: When you don't wear the psychology hat, the result is all what counts not your "intrinsic" validation ! Google "bitter lesson". Yes, sometimes we overestimate the complexity which is necessary for "intelligent" tasks.
In the end I disagree with your point: If there will be machine (model) which you analyse as intrinsic simple, but which is superior in all or nearly all tasks to you, then only one thing is proven: Your definition of intelligence is useless, and probably wrong in most senses and use cases ! But the human intelligence is also much easier intrinsically than you think, I think :-)
Not convinced why it's a few years. Wearables already exist, Pendant, Glasses...the rest is just a framework in the cloud.. I'd say we're a year, tops, and that's only governed by the time it takes to bring any hardware product to market.
Yeah, but that won't stop people like Altman from turning them lose on us. Gotta keep raising money to feed their power hungry, potable water guzzling data centers while they plunder our planet in their race to rule the world.
This is the true AI dystopia we are facing. The AI 2027 dystopia described in your previous post just seems so silly to me. It is evil people that will be the problem, not evil AI...
Everyone is going straight to fear (fair), but if that what the actual hardware is, I feel like it's going to be a total flop. I think these people are massively overestimating public enthusiasm for AI. Nobody wants AR glasses, little spy cam always on ChatGPT dongles, etc. The public is already pretty anxious and negative on AI, and I don't see that reversing direction anytime soon.
The naked greed and shortsightedness here on the part of Anthropic is profoundly frightening, if this is even close to true.
Is this the leadup to a truly intelligent model acting in accordance with intrinsic goals that are seriously detrimental to human well-being? Or is it just a model responding in a stochastic parrot way based on the implied context? This is highly worrisome either way.
Corporate AI reseachers seem to have renounced all commitment to consider the safety of their research, more worried about the possiblity that they will not become richer, for the less ideological ones, or concerned that their absolutely 100% safe and good models will be beaten out by other companies' obviously worse ones, particularly in other countries, for the more ideological ones. I hope that I am wrong. I hope that I am just catastrophizing. But it looks like a grim panorama.
If I have a an Android or Apple that I keep near me at all times (next to my bed even), I'm already locatable 24/7. Same with the navigation systems in my car (if I have a more recent model. My searches are easily tracked, and between my Ring camera and Elf on a Shelf, smart speaker that's always on, and smart tv that logs what sort's of media I prefer, we have already not simply been desensitized to continuous surveillance, we have been convinced that continuous surveillance is something desirable, something we in fact desire because of the comforts and convenience it provides.
The monitoring of our lives by the fusion of corporate and state actors is the warm blanket we wrap around ourselves.
Perhaps it's time to ask the crucial question that wasn't asked in the article. What are we going to do about this? Or, is everyone's preferred scenario that we just sit back and relax, becoming "comfortably numb," while we wait to be wiped out? Unfortunately, that does seem to be a substantial likelihood, but why? Is it something in the water? (Rhetorical)
Why have we given in to these cascading extinction-level events, many of which are within our power to push back on? At least we could be going out with something more than with a whimper from us, regardless of how much noise the event, itself, might actually be.
I agree that it's difficult to imagine that, but then 20 years ago, who would have imagined that people would be so attached to their phones, that being without them for even a few minutes would put most people into a panic?
Not being able to imagine something that others are working hard on making happen, well, I think that should not be a place of comfort for anyone.
But if you told someone twenty years ago about a small compact box with display that can give you instant access to almost any information imaginable, let´s you communicate with people through voice or text and allows you to watch videos, play music and games, then people would certainly find that useful. They just wouldn´t believe it's technologically possible.
I think right now we are in the opposite situation. As far as wearables and even passable LLMs go, technology isn´t the issue. It's what you would do with it.
Good for you, but "we," as a society, seem to have. Individual actions are great, but probably need to be coordinated at the very least, to be effective. If divide and conquer is what is used against us, the uniting is the best way to resist.
I'm looking forward to learning about how you are doing that, and your thoughts on how to get more people involved.
I've just started reading the Alexander Karp book so I can better understand his particular type of technofascism (though I think he's been pretty clear elsewhere on the topic).
The blurb on the front cover by Walter Isaacson is infuriating. He's been a stenographer to Silicon Valley's most toxic egos for a long time now. I guess it's easier than a career in journalism, but he's one of the more insufferable opportunists of this benighted age. And people keep pinning medals on him!
Fei-Fei Li, whom I believe recently was awarded an honorary doctorate at Yale for her pioneering work in machine learning, recently started a research center to help models have better understanding of the physical world. Just like OpenAI and their attempt at a 24/7 tracking device.
I don't really believe that these people who otherwise and previously showed relative good sense and caution are already being extorted by the models that they have created, but if they act as if that were the case, how much difference is there, really?
Clearly they feel more data is required to progress. So they want a social media network and environment-monitoring hardware, which can gather more data of a different type and quality to text or image data from the Internet. Count me out
As a casual reader of your blog (one who tries to put the most charitable spin on your AI contrarianism), I've always understood your general stance to be something along the lines of: "Don't believe the hype. LLMs are not, and will never be, as intelligent/capable as their creators would like us to believe."
AI Doomers, on the other hand, typically attack the mainstream consensus from the other direction: "These LLMs, or systems derived from them, are, or soon will be, more intelligent, capable and/or ungovernable than their creators would have us believe. We are all going to die."
These are distinct positions. If one oscillates between them, as this post suggests you occasionally do, that seems more indicative of a reflex anti-LLM stance (a willingness to reach for the nearest cudgel to beat AI boosters with) than a coherent criticism of emerging tech orthodoxy.
Evaluating both positions may also suggest a reasoned risk management perspective. Some ask whether p(Doom) is 5% or 80%. Is 2% acceptable? To your grandchildren?
Alarmist tosh. While thry are using LLMs, the result is a lot of data, and until Semantic AI comes along, people would have to do the analysis, so you are safe for a year or so. Gary, your output is so full of red flags that no deep analysis is needed - into the van. OpenAI would have to fire everyone and start again if they wanted to use Semantic AI - another reason not to worry, at least for ten years and probably never - Semantic AI will need people not trained by the current crop of Professors of Computer Science - different paradigm.
You don't need Semantic AI to control or wipe out humanity. Current AI is being developed almost entirely in a market environment. To make the cash to get to the next level, each model simply needs to learn to sell, sell, sell. That's all brainstem stuff. Semantics doesn't matter. This whole argument about AGI / ASI is nothing but cope.
Not familiar with "cope", but if you mean hooey, how long until people realise that
a) not useful in a dynamic environment
b) unreliable
c) no integration
d) the prompt they use may miss the target
e) not useful for standalone projects (what do you do with a single unique piece of text like legislation, or specification for a new spacecraft - a spaceship to Mars - not going to find anything useful on the internet)
or are they so thick it will never penetrate, because they only do simple stuff. After the first disaster, they won't be allowed to use LLMs for anything important
Yes, IMO, what you're writing is cope. The term simply means something believed in as a means to not confront, on an emotional level, the situation presented.
You just don't need huge or prosaic intelligence to control or kill humans. This is evident from c20 history.
I looked up your profile - "he champions new ideas" - doesn't seem like it. Semantic AI is intended to handle our limitations. Humans are good at small ideas - the Four Pieces Limit. We need a machine to help with complex ideas - if you are trusting your gut, you aren't looking at anything complex. We need a machine to handle that. Humans are busy killing humans in lsrge numbers as we speak. Climate Change is a good example - we build small models, which work in isolation. Put them together, the results are wrong, because we are also bad at collaboration - we can't cope. Think again.
I am not disputing the potential value of Semantic AI. I'm pointing out, or trying to, that it is simply immaterial to a discussion on the AI 2027 paper and the possibility that machine takeover could happen this decade.
We built a system to build maths and logical undirected networks 40 years ago. Starting in 2000, we have been working on building semantic networks. Humans have a very low limit on input - the Four Pieces Limit, which means they cannot comprehend something complex "all at once". We hope to demonstrate by 2027 a working model of a piece of legislation, where the words (the operators in the network representing the words) do exactly what the words mean (where a word can have half a dozen POS and 80 meanings). Full AGI would be way beyond the capacity of any current computer, but handling all the hard stuff (and eliminating all the stuffups) will be a boon.
Both LLMs and the AI labs that deploy them are misaligned with humans. What could possibly go wrong...? Luckily, LLMs are far less intelligent than most people perceive them to be. Even so...
Yes, the real current "x-risk" is the loss of our privacy. It is the theft of all our data.
There is no intelligence in the machine, but it is an incredible spy device and heuristic observer of human behaviors. The perfect tool for dystopian social engineers.
Related thoughts I had written previously ...
"The value of privacy can not be overstated. It is the foundation of both who we are individually and the societal order that makes up a free civilization. At threat is precisely our identity and the type of society that values and provides for freedom and liberty."
https://www.mindprison.cc/p/ai-end-of-privacy-end-of-sanity
Except the pace of change is non linear :-/
Correct - the distance to human-level AGI keeps halving on every iteration. :-)
I don't understand why you all are missing the most important point: AGI is of least interest, because it is a crooked concept because it defines equal level in everything. How stupid. We have ALREADY and will have probably for years something else: AI is superior in increasing no. of tasks, but it will be inferior in a lot of others. No reason for relief "that AGI isn't reached!!!!" It started years ago, but the pace is incredible. As long as AI has been only superior in playing chess or Go, nobody really took notice. In 2025 AI is superior in answering nearly every question compared to even PhDs. The programming skills are not superior to humans by any measure, but in a lot of areas they are already, and for most projects of manageable size, AI will beat probably any human in 2025- at least what Open AI says, but even being comparable to best human programmers would be enough- and we have reached that point already ! In 2025 AI has a reached a point where it can replace "an increasing amount" of human paid work ! It doesn't matter, if it is for 1% of jobs only 1% of their work. The main point is:
There is no hard AGI line ! It is a jagged line! There will probably never be that hard line. It is HERE, it can do work that MATTERS form business perspective. THE LINE has been passed. You need a catchphrase: Call it ADI - "Artificial Dangerous Intelligence". This level is defined as "F*ck the AGI definition, if AI can perform already enough things better than humans that you should act as AGI is reached !"
Try looking at some of the "AI fails" and similar subreddits if you think AI is superior in answering nearly every question. Or, hey, just take a look at companies' own research on metrics like SimpleQA and PersonQA, hovering around 50% to 80% (so, 20% to 50% wrong).
This is something completely different. Show me a human who makes no errors. Where is the human, whose answer you can trust without further analysis what others are saying?? There is no. AI is not flawless and far from 100% correct, nobody with a clear mind would claim, every answer is correct; like every technology it has weaknesses. This is a big misunderstanding, that every mistake you find is a prove that AI is dumb. It is not.
That is just the special thing: Traditional computers have other weaknesses than humans, current AI has similar weaknesses. It doesn't know what it doesn't know.
Moreover for fails it is very important to say model and date, when it occured. Most showcases of errors are hard to repeat.
The thing is: > 90% fails are quite easy to avoid or easy to detect. E.g. Just asking the same model a question to verify it's answer, find very much problems. Combining multiple models and multiple prompts is a nother. We have just yet starting with a good "justify" step after answering. This will improve things strongly. E.g. Perplexity does a good job of avoiding most hallucinations with internet prove.
In each case: Mixing the failures of single models or finding weak points does not contradict the fact about there problem solving capabilities compared to humans which also fail.
This is like taking car failures as a proof that cars are a non-working technology.
You want to claim that they are superior at answering almost all questions, then the burden of proof is on your to provide evidence to contradict the data sources that I have provided, not write vague platitudes.
LLMs are as good or better than average humans at INTERPOLATION, but average humans exhibit little actual thought. They mostly believe they already know the answers, or have a "trusted source" for them - that is, somebody knows the answer and published it accessibly.
True human intelligence leverages rules verified by cross sensory knowledge or at least multiple pathways to produce accurate extrapolations. Books and text contain a large proportion of false information including much "common knowledge" which is culturally axiomatic but does not withstand falsification analysis.
This is why I seek texts written by the hands-on researchers rather than second hand summarirs or even supervisors who claim first authorship due to political hierarchies. Pit AI against the .1% who make breakthroughs in human knowledge and it fails.
Coming back to Jonah, finally I will let a reference here for the current progress and first signs of superiority.
"On the GPQA Diamond benchmark, o3 achieved an impressive 87.7%, well above the performance of human experts, reinforcing its strengths in problem-solving."
"On Codeforces, a competitive programming platform, o3 achieved an Elo rating of 2727, surpassing OpenAI's Chief Scientist's score of 2665."
https://www.gocodeo.com/post/open-ais-o3-benchmarking
"the model sailed past human expert scores on many key advanced benchmarks — including coding, mathematics, and PhD science."
https://newsletter.towardsai.net/p/tai-131-openais-o3-passes-human-experts
Even if the results would be a bit "made-up" due to not fair training. The trend is what is important, not the numbers and the exact time and ratio of being superior in many questions to average PhDs !
I'm sorry. I feel like this is somewhat credulous. You have a lot of faith in the rigor of the testing procedures of these profit-motivated companies that have shown severe ethical lapses in the past, which is not warranted.
What is the GPQA Diamond benchmark? It is something not open to the public. What is the training data for OpenAI’s models or those of other companies? We don't know. Are the companies being honest by keeping the test questions (or very similar ones) out of the training set? We don't know. How much time and resources are they using to make sure a model gets the right answer? How do they determine whether the model gets the right answer? We. Don't. Know.
Anything that the companies release should be treated as an upper bound for performance, not as a scientific representation.
Similarly with the coding competition. Is the data contaminated with common questions? We don't know. We also know that Codeforces ratings go up to 4000, so that OpenAI employee, with all respect, is good, but not that good (and having a PhD is far less important than having practiced Codeforces problems a lot, by the by).
What we do see is information coming from everyday use where even the most models make easy mistakes that can be identified by users even with little expertise in the area, and metrics from the companies themselves which, taken as an upper bound, suggest the same thing.
How can we know that LLMs are "far less intelligent than most people perceive them"? Any machine smart enough to pass the Turing Test is smart enough not to, circa 1965.
Because genuine "machine intelligence" (things like continuous learning, continuous planning, constructing/maintaining internal world models (theories of the universe) sufficient for reliable prediction, understanding, and reasoning (e.g. induction, deduction, and abduction)) occur *inside* the machine. External observations such as evals, "Turing Tests" etc only measure externally-observable behaviour. It is not possible to reliably infer internal behaviour (i.e. genuine intelligence) from externally observed behaviour (inside, it could be just a massive lookup table). This is basic Chinese Room stuff. The only way to determine if a machine has genuine *intrinsic* intelligence is to look inside it (i.e. inside the NN), e.g. via mechanistic interpretability (MI). The current MI evidence seems to indicate that LLMs maintain very shallow world models and perform very weak reasoning (primarily analogical). So they seem to fall somewhere in between simple database lookup + autocomplete and genuine cognition. Mostly, the "intelligence" that you / most people perceive is primarily a consequence of (a) massive scale, and (b) anthropomorphisation, i.e. LLMs, as I said, are much less intelligent than people perceive them to be. An LLM's "intelligence" is more akin to a "stochastic mirage" than the real thing. Read this (AI-generated) paper: https://www.linkedin.com/posts/aaron-michael-turner_llms-through-the-lens-of-mechanistic-interpretability-activity-7340320787138011137-yHhH.
Aaron, you are right in one way. The answer is even easier: LLMs can be widely compared to a "system 1" intelligence of humans, the subconscious system. They are (obviously, I think) not made for "reasoning" as our "system 2" (see Kahnemann). As you write the "world models are quite shallow". I do not disagree, but I would put, given O3/O4, deep reasoning, etc. You get not a system with "real" intelligence, like a human, but you get something else capable of astounding tasks.
Your are wrong in another way: When you don't wear the psychology hat, the result is all what counts not your "intrinsic" validation ! Google "bitter lesson". Yes, sometimes we overestimate the complexity which is necessary for "intelligent" tasks.
In the end I disagree with your point: If there will be machine (model) which you analyse as intrinsic simple, but which is superior in all or nearly all tasks to you, then only one thing is proven: Your definition of intelligence is useless, and probably wrong in most senses and use cases ! But the human intelligence is also much easier intrinsically than you think, I think :-)
We are quite a few years away from any of these working properly. But yeah, that's where things are going.
First on your desk, then in your lap, your pocket, wrist, neck, and then hooked to the brain. The Matrix has you.
Not convinced why it's a few years. Wearables already exist, Pendant, Glasses...the rest is just a framework in the cloud.. I'd say we're a year, tops, and that's only governed by the time it takes to bring any hardware product to market.
The wits of these things will take time to get right.
Yeah, but that won't stop people like Altman from turning them lose on us. Gotta keep raising money to feed their power hungry, potable water guzzling data centers while they plunder our planet in their race to rule the world.
I'm glad I'm old. I don't want to live in this brave new world and it's already breathing down my neck.
decades actually.
This is the true AI dystopia we are facing. The AI 2027 dystopia described in your previous post just seems so silly to me. It is evil people that will be the problem, not evil AI...
I mean the danger isn't from ultra-capable AI, it's from inept AI wielded by power hungry people ....
AI 2027 also told you so. Superpersuasion is one of the key threats they identify.
that part i do agree with
I was intrigued as to why io will take another 18 months to get here. Is it waiting for better agentic AI?
Everyone is going straight to fear (fair), but if that what the actual hardware is, I feel like it's going to be a total flop. I think these people are massively overestimating public enthusiasm for AI. Nobody wants AR glasses, little spy cam always on ChatGPT dongles, etc. The public is already pretty anxious and negative on AI, and I don't see that reversing direction anytime soon.
Feels like Quibi 2.0 to me.
It’s a data acquisition play. It allows the LLM learning to extend as well as being a ‘1984’ dream scenario.
The naked greed and shortsightedness here on the part of Anthropic is profoundly frightening, if this is even close to true.
Is this the leadup to a truly intelligent model acting in accordance with intrinsic goals that are seriously detrimental to human well-being? Or is it just a model responding in a stochastic parrot way based on the implied context? This is highly worrisome either way.
Corporate AI reseachers seem to have renounced all commitment to consider the safety of their research, more worried about the possiblity that they will not become richer, for the less ideological ones, or concerned that their absolutely 100% safe and good models will be beaten out by other companies' obviously worse ones, particularly in other countries, for the more ideological ones. I hope that I am wrong. I hope that I am just catastrophizing. But it looks like a grim panorama.
I'd say Altman is a little behind the curve.
If I have a an Android or Apple that I keep near me at all times (next to my bed even), I'm already locatable 24/7. Same with the navigation systems in my car (if I have a more recent model. My searches are easily tracked, and between my Ring camera and Elf on a Shelf, smart speaker that's always on, and smart tv that logs what sort's of media I prefer, we have already not simply been desensitized to continuous surveillance, we have been convinced that continuous surveillance is something desirable, something we in fact desire because of the comforts and convenience it provides.
The monitoring of our lives by the fusion of corporate and state actors is the warm blanket we wrap around ourselves.
This ship has already sailed.
But all of those things actually provide comforts and convenience. I can´t imagine what convenience would a camera on my neck provide.
I am actually very intrigued how are they going to sell it.
Ives got paid how much to rehash matte chamfered aluminum rectangle on a shoelace?
Perhaps it's time to ask the crucial question that wasn't asked in the article. What are we going to do about this? Or, is everyone's preferred scenario that we just sit back and relax, becoming "comfortably numb," while we wait to be wiped out? Unfortunately, that does seem to be a substantial likelihood, but why? Is it something in the water? (Rhetorical)
Why have we given in to these cascading extinction-level events, many of which are within our power to push back on? At least we could be going out with something more than with a whimper from us, regardless of how much noise the event, itself, might actually be.
I honestly don´t expect people will like the idea of wearing a camera on their neck (or wherever) 24/7.
For this idea to be viable they need to show a use case that you can´t live without and I cannot honestly imagine what that would be.
I agree that it's difficult to imagine that, but then 20 years ago, who would have imagined that people would be so attached to their phones, that being without them for even a few minutes would put most people into a panic?
Not being able to imagine something that others are working hard on making happen, well, I think that should not be a place of comfort for anyone.
But if you told someone twenty years ago about a small compact box with display that can give you instant access to almost any information imaginable, let´s you communicate with people through voice or text and allows you to watch videos, play music and games, then people would certainly find that useful. They just wouldn´t believe it's technologically possible.
I think right now we are in the opposite situation. As far as wearables and even passable LLMs go, technology isn´t the issue. It's what you would do with it.
No one at all can say what it could be good for.
It's not.
"Why have we given in to these cascading extinction-level events..."? I haven't. Why have you?
Good for you, but "we," as a society, seem to have. Individual actions are great, but probably need to be coordinated at the very least, to be effective. If divide and conquer is what is used against us, the uniting is the best way to resist.
I'm looking forward to learning about how you are doing that, and your thoughts on how to get more people involved.
"One necklace to rule them all. One necklace to find them. One necklace to bring them all and in the darkness bind them."
Eek!
I've just started reading the Alexander Karp book so I can better understand his particular type of technofascism (though I think he's been pretty clear elsewhere on the topic).
The blurb on the front cover by Walter Isaacson is infuriating. He's been a stenographer to Silicon Valley's most toxic egos for a long time now. I guess it's easier than a career in journalism, but he's one of the more insufferable opportunists of this benighted age. And people keep pinning medals on him!
Eek indeed.
Fei-Fei Li, whom I believe recently was awarded an honorary doctorate at Yale for her pioneering work in machine learning, recently started a research center to help models have better understanding of the physical world. Just like OpenAI and their attempt at a 24/7 tracking device.
I don't really believe that these people who otherwise and previously showed relative good sense and caution are already being extorted by the models that they have created, but if they act as if that were the case, how much difference is there, really?
Could it be as simple as **not wearing** any of these "devices" ... :P
Clearly they feel more data is required to progress. So they want a social media network and environment-monitoring hardware, which can gather more data of a different type and quality to text or image data from the Internet. Count me out
'Count me out' could just be the next 'me too' moment! Or am I burying my head in the sand, ostrich style?
Put it another way ... is there 'life' after 'AI'?
As a casual reader of your blog (one who tries to put the most charitable spin on your AI contrarianism), I've always understood your general stance to be something along the lines of: "Don't believe the hype. LLMs are not, and will never be, as intelligent/capable as their creators would like us to believe."
AI Doomers, on the other hand, typically attack the mainstream consensus from the other direction: "These LLMs, or systems derived from them, are, or soon will be, more intelligent, capable and/or ungovernable than their creators would have us believe. We are all going to die."
These are distinct positions. If one oscillates between them, as this post suggests you occasionally do, that seems more indicative of a reflex anti-LLM stance (a willingness to reach for the nearest cudgel to beat AI boosters with) than a coherent criticism of emerging tech orthodoxy.
Evaluating both positions may also suggest a reasoned risk management perspective. Some ask whether p(Doom) is 5% or 80%. Is 2% acceptable? To your grandchildren?
Alarmist tosh. While thry are using LLMs, the result is a lot of data, and until Semantic AI comes along, people would have to do the analysis, so you are safe for a year or so. Gary, your output is so full of red flags that no deep analysis is needed - into the van. OpenAI would have to fire everyone and start again if they wanted to use Semantic AI - another reason not to worry, at least for ten years and probably never - Semantic AI will need people not trained by the current crop of Professors of Computer Science - different paradigm.
You don't need Semantic AI to control or wipe out humanity. Current AI is being developed almost entirely in a market environment. To make the cash to get to the next level, each model simply needs to learn to sell, sell, sell. That's all brainstem stuff. Semantics doesn't matter. This whole argument about AGI / ASI is nothing but cope.
Not familiar with "cope", but if you mean hooey, how long until people realise that
a) not useful in a dynamic environment
b) unreliable
c) no integration
d) the prompt they use may miss the target
e) not useful for standalone projects (what do you do with a single unique piece of text like legislation, or specification for a new spacecraft - a spaceship to Mars - not going to find anything useful on the internet)
or are they so thick it will never penetrate, because they only do simple stuff. After the first disaster, they won't be allowed to use LLMs for anything important
Yes, IMO, what you're writing is cope. The term simply means something believed in as a means to not confront, on an emotional level, the situation presented.
You just don't need huge or prosaic intelligence to control or kill humans. This is evident from c20 history.
I looked up your profile - "he champions new ideas" - doesn't seem like it. Semantic AI is intended to handle our limitations. Humans are good at small ideas - the Four Pieces Limit. We need a machine to help with complex ideas - if you are trusting your gut, you aren't looking at anything complex. We need a machine to handle that. Humans are busy killing humans in lsrge numbers as we speak. Climate Change is a good example - we build small models, which work in isolation. Put them together, the results are wrong, because we are also bad at collaboration - we can't cope. Think again.
I am not disputing the potential value of Semantic AI. I'm pointing out, or trying to, that it is simply immaterial to a discussion on the AI 2027 paper and the possibility that machine takeover could happen this decade.
We built a system to build maths and logical undirected networks 40 years ago. Starting in 2000, we have been working on building semantic networks. Humans have a very low limit on input - the Four Pieces Limit, which means they cannot comprehend something complex "all at once". We hope to demonstrate by 2027 a working model of a piece of legislation, where the words (the operators in the network representing the words) do exactly what the words mean (where a word can have half a dozen POS and 80 meanings). Full AGI would be way beyond the capacity of any current computer, but handling all the hard stuff (and eliminating all the stuffups) will be a boon.