If you believe in innateness, you don’t believe in learning If you believe in learning, you should oppose innateness The more things are learned, the less we need innateness If you believe in innateness, you have to believe in innate, domain-specific knowledge
Case in point: deep learning architectures are designed. E.g. BERT is bi-directional, GPT uni-directional. This difference is not learned but preset ('inborn') to influence learning.
But it is interesting to ask if critical aspects of compositional cognition, e.g. the 'logistics of access' it requires, can be learned from a more basic architecture or need to be preset.
Hi Gary, excellent article that lays out the two 'sides' :) Indeed, nurture won't be useful without nature.
Also - Bloom's Taxonomy offers a quite useful, graduated/hierarchical list of capabilities, that can serve to create tests against which to assess AI mastery. AI thus far has been stuck at the bottommost level :) :(
Also, seems to me that human learning stands apart from all others', on account of our innate abilities to represent happenings directly, ie gain body-based "experience", AND to represent things (direct experiences, objective knowledge...) symbolically as well. This duality lets us glide back and forth, lets us symbolize our knowledge and experience for others to pick up, and conversely, lets us benefit from others' symbolizations (going back to 1000s of years!). Other animals seem more limited in the 'direct <-> symbolic' mapping.
I agree that no rational person who is even remotely aware of findings in neuroscience and cognitive pyschology would disagree that both learning and innateness are part of the answer. I also like how you state the ovious, that learning cannot be explained without admitting that the "learning mechanisms" (or at least many/most?) must be innate - lest we run into an infinite regress. That's an excellent point.
To me, the debate is not whether we have both, learning and innateness. To me the question is how much of each, and what kind of each - i.e., what is it that we learn, and what is that we do not - and cannot, in a lifetime, learn. These are the questions that need to be answered. A chimpanzee living with humans will never be more intelligent than a baby left alone in the jungle. The question thus is what is it that is innate? What mechanisms and what universal logic seems to be built in that ALLOWS US to learn the rest.
"Why am I into both learning and innateness? In large part because I care about what make humans unique."
One take on what makes humans unique...
1) We are made of thought.
2) Thought operates by dividing a single unified reality in to conceptual parts.
3) This conceptual division process is the source of both our genius and insanity.
GENIUS: We can rearrange the conceptual parts in our minds to create visions of reality which don't yet exist, that is, we can be creative.
INSANITY: This thought generated conceptual division process creates a human experience where we feel divided from reality, divided from each other, and even divided within our own minds. This experience of isolation generates fear, which is in turn the source of most human problems.
The best example of this marriage between genius and insanity may be nuclear weapons. What makes us unique? No other species possesses both the brilliance and insanity necessary to engineer it's own extinction.
We are made of thought. Thought operates by a process of division. And the rest of the human story flows from there. This is the innate foundation upon which AI is being developed.
The mechanism of autonomous learning is innate. We can consider this as a kind of knowledge, represented by algorithms and implemented as "hardware". All other knowledge is acquired in the learning process.
Agreed that we should expect the human brain and body to be primed for learning and that there is no reason to pretend that we can insulate learning from that priming. The language of nativism and innateness is something I don't tend to embrace (too much confusion ensues in my neck of the intellectual woods).
I am curious where Gary stands on Michael Tomasello's interventions into language acquisition/human communication.
It may just be me but I never understood the importance of this debate. Isn't it obvious that in order for the brain to learn anything, it must first exist? There can be no learning without an existing learning mechanism designed for that purpose. Why is there a debate about this? Should not the discussion be about what is the best innate mechanism for generalized intelligence?
In this light, I see nothing in either deep learning or symbolic AI that can properly generalize. The ability to generalize is fundamental to intelligence and must be innate to it. That is, a truly intelligent system must be designed from the ground up with generalization in mind. This includes the design of its sensors and effectors. Many biologists are aware of the complex design of biological sensors, especially the retina and the cochlea. Nothing must be taken for granted.
I would like to add another myth: "Innateness" is a word that means something useful rather than simply a stand in for: 'We are not sure what is going on, really, other than there is something going on beyond just gathering data and creating layers and connections'. In other words, stating that there is something "innate" is not much more sophisticated than what Descartes called "soul" or Kant "noumenal". I would like to suggest that the beef with "innateness" starts and ends with the fact that people who can't define it insist that it has a precise, operationalizable meaning, when it just doesn't.
Hi Gary,
I fully agree.
Case in point: deep learning architectures are designed. E.g. BERT is bi-directional, GPT uni-directional. This difference is not learned but preset ('inborn') to influence learning.
But it is interesting to ask if critical aspects of compositional cognition, e.g. the 'logistics of access' it requires, can be learned from a more basic architecture or need to be preset.
Best,
Frank van der Velde
Hi Gary, excellent article that lays out the two 'sides' :) Indeed, nurture won't be useful without nature.
Also - Bloom's Taxonomy offers a quite useful, graduated/hierarchical list of capabilities, that can serve to create tests against which to assess AI mastery. AI thus far has been stuck at the bottommost level :) :(
Also, seems to me that human learning stands apart from all others', on account of our innate abilities to represent happenings directly, ie gain body-based "experience", AND to represent things (direct experiences, objective knowledge...) symbolically as well. This duality lets us glide back and forth, lets us symbolize our knowledge and experience for others to pick up, and conversely, lets us benefit from others' symbolizations (going back to 1000s of years!). Other animals seem more limited in the 'direct <-> symbolic' mapping.
Still reading this but another myth might be:
Deep learning is so powerful it will learn whatever innate knowledge it needs to know anyway.
I agree that no rational person who is even remotely aware of findings in neuroscience and cognitive pyschology would disagree that both learning and innateness are part of the answer. I also like how you state the ovious, that learning cannot be explained without admitting that the "learning mechanisms" (or at least many/most?) must be innate - lest we run into an infinite regress. That's an excellent point.
To me, the debate is not whether we have both, learning and innateness. To me the question is how much of each, and what kind of each - i.e., what is it that we learn, and what is that we do not - and cannot, in a lifetime, learn. These are the questions that need to be answered. A chimpanzee living with humans will never be more intelligent than a baby left alone in the jungle. The question thus is what is it that is innate? What mechanisms and what universal logic seems to be built in that ALLOWS US to learn the rest.
Isn't that the issue?
And thanks for another great article.
Marcus writes..
"Why am I into both learning and innateness? In large part because I care about what make humans unique."
One take on what makes humans unique...
1) We are made of thought.
2) Thought operates by dividing a single unified reality in to conceptual parts.
3) This conceptual division process is the source of both our genius and insanity.
GENIUS: We can rearrange the conceptual parts in our minds to create visions of reality which don't yet exist, that is, we can be creative.
INSANITY: This thought generated conceptual division process creates a human experience where we feel divided from reality, divided from each other, and even divided within our own minds. This experience of isolation generates fear, which is in turn the source of most human problems.
The best example of this marriage between genius and insanity may be nuclear weapons. What makes us unique? No other species possesses both the brilliance and insanity necessary to engineer it's own extinction.
We are made of thought. Thought operates by a process of division. And the rest of the human story flows from there. This is the innate foundation upon which AI is being developed.
The mechanism of autonomous learning is innate. We can consider this as a kind of knowledge, represented by algorithms and implemented as "hardware". All other knowledge is acquired in the learning process.
Learning is signal transduction in biology or System 1 in cognitive sciences.
Agreed that we should expect the human brain and body to be primed for learning and that there is no reason to pretend that we can insulate learning from that priming. The language of nativism and innateness is something I don't tend to embrace (too much confusion ensues in my neck of the intellectual woods).
I am curious where Gary stands on Michael Tomasello's interventions into language acquisition/human communication.
It may just be me but I never understood the importance of this debate. Isn't it obvious that in order for the brain to learn anything, it must first exist? There can be no learning without an existing learning mechanism designed for that purpose. Why is there a debate about this? Should not the discussion be about what is the best innate mechanism for generalized intelligence?
In this light, I see nothing in either deep learning or symbolic AI that can properly generalize. The ability to generalize is fundamental to intelligence and must be innate to it. That is, a truly intelligent system must be designed from the ground up with generalization in mind. This includes the design of its sensors and effectors. Many biologists are aware of the complex design of biological sensors, especially the retina and the cochlea. Nothing must be taken for granted.
PS. Even insects generalize.
I would like to add another myth: "Innateness" is a word that means something useful rather than simply a stand in for: 'We are not sure what is going on, really, other than there is something going on beyond just gathering data and creating layers and connections'. In other words, stating that there is something "innate" is not much more sophisticated than what Descartes called "soul" or Kant "noumenal". I would like to suggest that the beef with "innateness" starts and ends with the fact that people who can't define it insist that it has a precise, operationalizable meaning, when it just doesn't.
1. "best nearby candidate" - almost a quote from my definition for intelligence.
2. "both learning and innateness" - newborns do not know about innateness, but they have brains and may learn about innateness eventually.
3. minor note - "humans unique" - is it about both sapiens and neanderthalensis?