Is "it's complicated" a civilised way to say 'clusterfuck'? Or might all of thus mean that OpenAI has handed Microsoft the means to fill whatever mini-'moat' OpenAI had? Did OpenAI give away whatever crown jewels they had in that Microsoft deal that got them the compute they needed? Definitely intriguing.
The '1 trillion parameters' claim for GPT-4 comes out of internet guesswork as far as I have been able to find out, nobody really knows, and 'Open'AI isn't telling. I suspect it is bogus, though it is theoretically possible. And what is more: the *number* of parameters isn't important, the *volume* (number x average size) is. Gemini for instance, is known to partly use "int8" parameters (Google told us) as the engineers have optimized to more but less precise parameters ones so they can get more (but less precise) 'optima'.
Yeah, especially with mixture-of-experts models, and with RAG and use of tools, the number of parameters stops being a useful metric, as there's a lot more than a single neural net.
I am reasonably convinced that all of this together is going to produce useful tools. I also suspect the inefficiencies of the whole architecture will result in financial and environmental issues. The messianic hoopla around it is rather irritating.
The question is how to properly learn from data that is very poorly structured and for which it is very hard to build models.
Extracting patterns from a very large amount of examples looks to me like a very solid first step. Then we need to solve the smaller problems of verification, custom strategies depending on context, interfacing with frameworks that know about modeling, etc.
Making machines understand what words mean is a hard problem. For simple-enough applications just going through the motions and having strategies to check work as people do it, with external tools, can hep.
My hope is that chatbots can function as a translation layer, with the heavy lifting done by other tools that do honest modeling.
Microsoft is big. Long-term, it will want to have total control. 10 billion it invested in OpenAI for short-term gains is not much for Microsoft. It makes sense it wants to chart its own path.
Tbh if there was someone I trust less than OpenAI, it is Microsoft. If I had one wish, I would delete AI. But failing that, deleting Microsoft from AI alone would be something as they have repeatedly demonstrated incredible lack of safety beyond even OpenAI and this also with internal information about them.
Given that every single copilot Microsoft has built is largely because of GPT-4, and that they will likely get early access to GPT 5, it does make you wonder who is getting the better side of the deal.
Mainly though it makes you wonder about the antitrust implications and all the lawsuits they are going to face as one unit, where most startups Will survive that's sort of litigation, OpenAI can take risks that Microsoft cannot and survive the onslaught due to their lawyers and lobbying.
This relationship is highly illegal, but protected due to the factors that you mentioned.
I'm thinking that maybe Satya checks to see if any silverware is missing after he's had Sam over to the house for dinner. Can't be too careful with these tricksters.
As I understood the MS investment, it was actually in discounted / free use of their AI infrastructure, so that no cash passes. I may be wrong, but that was my understanding.
To me at least it seems more likely they'll just drop OpenAI once it becomes clear there is no further (economical!) upside to supporting them. If and when that would happen is up for debate, of course...
Microsoft is hedging its bets. If OpenAI keeps on being the lead, Microsoft can milk that. If the industry catches up and even pulls ahead, as it is very likely with Google, Microsoft will not want to be bound to OpenAI.
Is "it's complicated" a civilised way to say 'clusterfuck'? Or might all of thus mean that OpenAI has handed Microsoft the means to fill whatever mini-'moat' OpenAI had? Did OpenAI give away whatever crown jewels they had in that Microsoft deal that got them the compute they needed? Definitely intriguing.
Relation"shop" haha is that deliberate?
ooh that would have been clever if deliberate
You are nothing if not clever!
The '1 trillion parameters' claim for GPT-4 comes out of internet guesswork as far as I have been able to find out, nobody really knows, and 'Open'AI isn't telling. I suspect it is bogus, though it is theoretically possible. And what is more: the *number* of parameters isn't important, the *volume* (number x average size) is. Gemini for instance, is known to partly use "int8" parameters (Google told us) as the engineers have optimized to more but less precise parameters ones so they can get more (but less precise) 'optima'.
Yeah, especially with mixture-of-experts models, and with RAG and use of tools, the number of parameters stops being a useful metric, as there's a lot more than a single neural net.
I am reasonably convinced that all of this together is going to produce useful tools. I also suspect the inefficiencies of the whole architecture will result in financial and environmental issues. The messianic hoopla around it is rather irritating.
We can make do without messianic stuff.
The question is how to properly learn from data that is very poorly structured and for which it is very hard to build models.
Extracting patterns from a very large amount of examples looks to me like a very solid first step. Then we need to solve the smaller problems of verification, custom strategies depending on context, interfacing with frameworks that know about modeling, etc.
And of course, being able to reason and understand what the tokens actually refer to in the real world.
Making machines understand what words mean is a hard problem. For simple-enough applications just going through the motions and having strategies to check work as people do it, with external tools, can hep.
My hope is that chatbots can function as a translation layer, with the heavy lifting done by other tools that do honest modeling.
Microsoft is big. Long-term, it will want to have total control. 10 billion it invested in OpenAI for short-term gains is not much for Microsoft. It makes sense it wants to chart its own path.
Tbh if there was someone I trust less than OpenAI, it is Microsoft. If I had one wish, I would delete AI. But failing that, deleting Microsoft from AI alone would be something as they have repeatedly demonstrated incredible lack of safety beyond even OpenAI and this also with internal information about them.
Given that every single copilot Microsoft has built is largely because of GPT-4, and that they will likely get early access to GPT 5, it does make you wonder who is getting the better side of the deal.
Mainly though it makes you wonder about the antitrust implications and all the lawsuits they are going to face as one unit, where most startups Will survive that's sort of litigation, OpenAI can take risks that Microsoft cannot and survive the onslaught due to their lawyers and lobbying.
This relationship is highly illegal, but protected due to the factors that you mentioned.
I'm thinking that maybe Satya checks to see if any silverware is missing after he's had Sam over to the house for dinner. Can't be too careful with these tricksters.
“I have heard of monogamy, polygamy, polyandry, polyamory, and ethical non-monogamy, to name a few, but this is just … complicated.”
It’s an orgy. Thing about orgies is that people usually go home afterwards and refuse to recognize each other on the street.
https://en.wikipedia.org/wiki/Embrace,_extend,_and_extinguish
I don’t understand the not for military thing. Silicon Valley was started with military orders during the Cold War.
As I understood the MS investment, it was actually in discounted / free use of their AI infrastructure, so that no cash passes. I may be wrong, but that was my understanding.
It was nothing personal, they both were just doing their jobs ;)
What would you say are the odds that OAI eventually gets eaten by Microsoft, similar to Inflection?
To me at least it seems more likely they'll just drop OpenAI once it becomes clear there is no further (economical!) upside to supporting them. If and when that would happen is up for debate, of course...
Microsoft is hedging its bets. If OpenAI keeps on being the lead, Microsoft can milk that. If the industry catches up and even pulls ahead, as it is very likely with Google, Microsoft will not want to be bound to OpenAI.