What might our minds be, if not computers?
Lecture notes
Lecture notes
By the end of this week, you'll have knowledge of powerful tools - Bayesian models of the mind, reinforcement learning for modeling human decision-making, and how our minds might respond to different drugs. But there's a risk here: with all these tools, you can get disoriented. You might begin to miss the forest for the trees. When choosing between exact tools, we may lose track of the bigger questions that drove the creation of these tools and that drive your research curiosity.
So when you're sitting at your laptop two or three months from now, thinking of your research idea, what I hope to give you in this lecture is a map. It's not a literal map, but it is a way of orienting towards cognitive science - the sciences of the mind. I'm going to do this by talking about the metaphors that have animated researchers who have studied the mind, and the questions that have come with those metaphors.
In studying the mind, there are three metaphors we're going to dive into - three deep research traditions that have shaped cognitive science since the 1950s:
The mind as a computer
The mind as a distributed neural network
The mind as a dynamic embedded system
With these come questions about how the mind manipulates symbols as a computer, how it recognizes patterns as a distributed neural network, and how it interacts with the world as an embedded system - what is the border between the mind and the world?
You might be wondering: why metaphors? Why should we care about metaphors? The way that we think, the way that we reason, the way that we talk is deeply metaphorical. It's metaphorical in how we reason with each other, one on one. If I ask you, "Do you see what I mean?" - as though ideas were things you could see. "Are you following my reasoning?" - like you can follow something through space. "Do you grasp this idea?" - as though an idea is like a package.
Let me take you through a brief history of how metaphors have shaped our understanding of ourselves:
About 7,000 years ago, the wheel was invented. A couple thousand years later, scholars in India declared that all of our existence is like a wheel turning round and round - the seasons come and go revolving, life and death follows each other. Everything is like a wheel, the grand wheel of becoming.
In ancient Greece, around 700 BC, they invented the wax tablet for keeping records of taxes. And Aristotle, reflecting on the mind and memory, declared it's like a wax tablet - memories imprint just like marks made on a wax tablet.
In the 1600s, hydraulic tubes are invented for water fountains in France. Descartes, the philosopher and mathematician, declares all our nerves are but tubes carrying animal spirits throughout our bodies.
And of course, in 1945, ENIAC, the first digital computer is invented. And what do you think the cognitive scientists declare? That leads us nicely to our first orienting metaphor
To set the scene a bit - as the first computers were being invented, how were people understanding the mind? No one really thinks like this now, but at the time, the dominant way of trying to understand our psychology was behaviorism. B.F. Skinner, who was a professor at Harvard, and other behaviorists famously thought we can't understand anything about the internal processes of the mind - we just have to look at the behavior and the functional output of the organism.
Skinner would do things like create this box - this operant conditioning box - where he would change the input stimuli and treat the rat's brain like a black box. If I shock it, if I give it food, how is it going to respond? And he would characterize closely the behavior of the rat in response to the stimuli.
It's against this backdrop that the computer is invented. And with the computer, there's a group of scientists - mathematicians, computer scientists, linguists - who are really frustrated with how the behaviorists just think you can't look inside the mind. Having been educated in mathematics and seeing that this mind-like thing, the computer, has individual components, they start what we now call the cognitive revolution.
Herbert Simon (a professor at CMU), and Marvin Minsky (the creator of the MIT AI Lab) had this shared hypothesis that both our brains and computers are fundamentally the same - they're physical symbolic systems. They used math to support this idea. Our brains and computers can run programs - they're Turing complete, which means that if you can translate any task into a computer problem, we can run and complete it.
The emphasis here is on the symbolic nature of our problem-solving. Rather than thinking about just pure behaviors, they're interested in manipulating symbols - A and B, and using operators to change those around.
Let me give you a practical demonstration of how these cognitivists think. One of the classic things they would study is called the Tower of Hanoi.
[Try a practicalÂ
The questions that naturally came with viewing the mind as a serial computer were:
How can physical systems manipulate symbols to solve problems?
How could you translate problems into the language of operators and variables?
What are the range of symbolic programs that we are using to learn languages, memory, and recognize patterns?
But there were some scientists who were frustrated with this approach. They recognized something was missing in this analogy of the brain as a serial computer. What do you think might be missing from this view of the mind as a serial computer?
By the 1960s, scientists had raised some crucial questions about the computer metaphor. One major issue was biological plausibility - we knew that the brain wasn't doing serial computation. There were billions of neurons working in parallel. So how could everything happen serially? Surely not everything's serial.
And then there was a more philosophical question: Okay, so you say that everything is like computer states. But how does the feeling that I get when I look at grass come from these symbols? What does experience have to do with computer states?
This brings us to the second group of people - the connectionists. Part of the backdrop here is that we actually had much better computers by this point. Back in the fifties, Simon and Newell ran their program on a device that had a kilobyte of memory. Now we had machines that had a whole megabyte of RAM! Because we had a whole megabyte of RAM, we could actually start to think about slightly more complex programs - programs that could even be parallelized.
Two key figures emerged here. On the right is Geoffrey Hinton - the English-Canadian computer scientist who we now call the Father of Deep Learning (and who recently received the Nobel Prize in Physics for his contributions). On the left is his mentor, David Rumelhart. They both met at UC San Diego, and their key contribution was in taking this idea of the perceptron and stacking it.
Let me explain the perceptron, or artificial neuron. If you look at how it works, there are multiple different inputs - these x1, x2, x3 that get summed together. Then a nonlinearity function is applied to it (like sigmoid or relu - don't worry about these terms if you haven't heard them). We call it a neural network because if you squint a little bit, you can see how it mirrors real neurons - how neurons have a single axon going outwards and dendrites leading into the neuron. The inputs (x1, x2) are like the inputs to the dendrites, and the axon is going out. Obviously, there's not a summing function in a particular neuron, but it's closer to being biologically plausible.
What Hinton and Rumelhart did was they took these neurons and stacked them to make a neural network. Each circle represents a unit, and it becomes a network because they're all connected to each other - the output of one goes into the input of another.
[Practical demonstration of parallel distributed processing]
So what Hinton and Rumelhart showed through these kinds of demonstrations is how computation can happen in a really distributed way. It's not just one person serially solving tasks - every neuron (in our demonstration, each of you was like a neuron) has input into whether something is right or wrong. Then we reweight all of the "votes" based on how right or wrong they are.
You can see how different this is than the process of trying to solve the Tower of Hanoi. When we do this over time, we typically see what's called a loss function graph. That's just an overall measure of how well the network is predicting the responses - how well, if we combine all of your outputs, it predicts the responses. What we find is that once you do hundreds, thousands of iterations, the loss goes down over time and the network gets better.
The core questions that animate this type of approach - this distributed neural network approach - are quite different from the cognitivist tradition. Instead of explicit rules like in the Tower of Hanoi, we ask: How do these distributed minds learn these complex patterns? One solution is what we just described through parallel distributed learning. And then the second question is: How can these patterns be represented? Neural networks suggest you have these neurons that get really good at predicting specific things - like recognizing cats.
But even with neural networks, some questions remained. As some scientists began to ask: Okay, so you have this neural network, but what does it have to do with the feeling I get when I look at grass? And more fundamentally - why do we think that the neural network is just in your brain?
Are neurons just in your skull? We now know that there's neurons actually in our hearts and in our stomachs. There's motor neurons through our arms. So it's not just a brain thing.
This brings us to the tradition of embodied cognition. It's currently the most diverse approach in terms of methodology - not as unified as the connectionists. Three key figures represent different angles of this approach:
Francisco Varela, a neurobiologist and somewhat of an anthropologist who studies frog cells and frog brains
Rodney Brooks, a robotics professor at MIT who uses this embedded systems approach to build better robots
David Chalmers, a philosopher
Let me make this concrete with an example - the Roomba. Both the cognitivists and the connectionists still had this assumption that the patterns and learning happened somehow in the brain. But what the embodied cognition researchers emphasized was different. Take the first iteration of the Roomba - it doesn't have a map of the environment. A cognitivist would want to specify everything and give the Roomba a model of the world it's going to operate in, but the Roomba doesn't do any of that.
Instead, it just goes straight, and then it bumps into something, and then it turns left. Then it goes again, bumps into something else, turns left - it basically wanders almost somewhat randomly. But it does so in a random enough way that it ends up covering the entire ground and cleaning your whole floor. It's actually doing something quite radical if you think about it - the Roomba is using the world as a computer. Rather than having to reference object variables and distances, it can just bump into a table and a chair and find out.
This raises one of the core questions that orients embodied cognition: Where does the mind end and the world begin? And how do they interact?
Let me share a famous thought experiment from David Chalmers. Imagine you meet two people at a party - a woman named Inga and an older man named Otto. Otto tells you he has Alzheimer's, so he writes your name down in his notebook. A few days later, you bump into both of them. Inga sees you, pauses for a second, consults her memory, and remembers your name - "Hey Max!" Just down the block, Otto sees you, doesn't quite remember your name, pulls out his notebook, looks it up - "Oh, it's Max, right?"
Here's the question: What is the fundamental difference between Otto's notebook and Inga's memory? Functionally, they're serving an identical purpose. Is there a principled way we can draw a distinction? It feels somehow different because it's "inside" Inga's mind, but in what way? We wouldn't be able to distinguish between Inga and Otto if we were just interacting with them through an interface. The argument Chalmers makes is that with Otto's notebook, his mind is being extended into the environment. This challenges the idea that memory just ends with the skull - they call this the extended mind hypothesis.
[The lecture describes an interactive experiment where participants use a pen to feel the texture of a desk, and then use the desk to investigate the properties of the pen]
What's really interesting about this experiment, which I borrowed from the philosopher-cognitivist John Becky, is that it demonstrates extension. When you're investigating the desk, the pen naturally becomes an extension of your body - your mind extends almost like it's an extra finger. They've done many experiments with people in VR learning to manipulate a new limb that actually feels like their limb, or people who lose their legs taking ownership over their mechanical leg.
When we swapped to investigating the pen using the desk, you were actually co-investigating with the environment. In terms of embodied cognition, you and the desk were interacting to investigate the pen - without the desk, you couldn't have done that investigation. This all points to the idea that the border between our minds and the world is more flexible than we might initially think.
So why are we talking about all these metaphors? When we have all these tools - neural networks for modeling the brain, probabilistic processes - we can get disoriented. When you're debugging a line of Python in your terminal, you can start to lose track of: What am I orienting towards? What am I assuming about the brain? What questions am I trying to ask?
All these metaphors - brain as computer, brain as neural net, brain as embedded system - are a map to help orient us in the science of mind. They've driven what we now call cognitive science, and they're still alive in various contexts. Probabilistic models have different threads you'll recognize - connectionist ideas in how they learn and update, cognitivist insights in how they frame problems as probabilities. Reinforcement learning, too, with its notion of an agent in an arena, draws heavily from cognitive-style representation.
These metaphors can help you investigate hard questions that no one really knows the answer to. For example, when you choose to model depression using a neural network, you're assuming this metaphor of the brain as a neural network, and you're assuming depression is like a brain problem. But is it just a brain problem? Or is it a multi-agent problem? An environmental problem? Is depression something just in the brain, or is it about interaction with the world - a social problem?
What about the meanings of stories too? Some people can be broke on the streets, struggling to find their next meal, but be beaming incredibly happy - they understand their struggle is meaningful. Others might be very comfortable and still very unhappy. There's something really important about meaning there that we might lose track of. That's still an open thread in cognitive sciences - where does meaning come in?
The questions I'd like to leave you with are: As you do your research, what metaphors are you seeing in mind? What questions are driving you?