How visual learning happens in the brain

The wiring and rewiring of the brain never ends. Neural pathways are constantly being reshaped as we interact with the world and learn new things. At MIT’s McGovern Institute for Brain Research and York University in Toronto, Ontario, scientists are combining detailed analysis of brain activity with computational modeling to better understand that change.McGovern Institute postdoc Lynn Sörensen, McGovern investigator and MIT Professor James DiCarlo, and York University Assistant Professor Kohitij Kar, worked together to compare what happened when animals and an artificial neural network with brain-like architecture were trained to visually identify the same objects. As the model’s performance improved, it reorganized itself in ways that closely paralleled changes the team detected in the animal brains.Their open-access work, reported July 8 in the journal Nature Communications, shows how changes in visual processing support animals’ ability to learn to discriminate new kinds of objects. By modeling these changes, the researchers hope to better predict how training reshapes perception, which could one day inform educational strategies for a wide range of learners.Subtle changesLearning about a new object calls on many parts of the brain. Visual-processing areas work together to make sense of information taken in through the eyes, then communicate with other brain areas to give the visual information meaning and guide behavior. Multiple parts of this system likely change during learning, and the research team wanted a clearer understanding of how that change is distributed.Neuroscientists have debated how much change occurs in the brain’s visual-processing areas when an animal learns to recognize new objects. Some suspected that visual-processing pathways remain largely unchanged during learning to avoid broadly disrupting visual perception, but others have reported changes in activity within dedicated visual-processing areas with this kind of learning in humans and other primates.  To take a closer look, the team focused on neural activity in a key component of the brain’s visual object-processing network, the inferior temporal (IT) cortex. By the time visual information reaches the IT cortex, key object features are clearly represented — so much so that it’s possible to “decode” what object the subject is seeing and even predict what errors it’s likely to make in identifying it, simply by analyzing patterns of neural activity there.The team recorded neural activity in the IT cortex from animals as they looked at and identified images of objects. Some of the animals were untrained, so the images they saw had little meaning to them. Others had already learned to identify similar objects, so they could usually discriminate between elephants, chairs, and other select objects, even when those objects were presented at different sizes, from different angles, or against different backgrounds than the ones they had seen before.The broad pattern of activity in the IT cortex was largely similar in trained and untrained animals, suggesting that learning had not dramatically rewritten this high-level visual representation. Still, the group found subtle but reliable differences in the way neurons in the IT cortex responded to images in animals that had learned to recognize the kinds of objects they were shown, compared to the untrained animals.Modeling learningThe group turned to computational models to investigate how those modest changes might contribute to learning. Sörensen trained a suite of artificial neural networks whose internal components had been mapped to the IT cortex to identify the same categories of objects the animals had seen. The models were designed to learn using gradient descent, meaning they continually improved their accuracy by adjusting their parameters in response to errors.Only some of the animal models showed learning behavior that matched that of the subjects. In those that did, the IT-like stage changed in ways that resembled the learning-related changes the researchers had observed in the IT cortex of trained animals.While gradient descent is commonly used to train artificial intelligence, it is generally considered biologically implausible as a direct model of how the brain learns. The researchers say the strong match in learning effects between the animals and their model demonstrates that these kinds of artificial neural networks can offer insights into biological learning at a useful level of abstraction, even if the brain does not learn in the same way.“This shows that you can actually build in silico versions of future experiments,” Sörensen says. “I think that gives us this playground of asking ‘what if’ questions — and potentially predicting new things that go beyond the experimenter’s intuition.”Most of the changes that allowed for learning in the model occurred outside of the IT cortex. “This tells us that there is a lot between the area we recorded from and the final behavioral readout that needs to change during this process,” Kar says. He adds that the team’s model will be useful as researchers look more deeply into how downstream brain areas contribute to learning.The researchers stress that their study allowed more granular measurements of brain activity than would be possible in humans, and because the animal brains are organized similarly to our own, their experiments have direct relevance to human learning. They say understanding the impact of plasticity in the subjects’ IT cortex could help researchers design new learning strategies for humans.“Our prior conceptual working model of you learning new objects was that your brain makes changes to synaptic connections that are largely downstream of your visual system, so you don’t destroy your visual system,” says DiCarlo, who is also the Peter de Florez Professor of Brain and Cognitive Sciences and director of the MIT Siegel Family Quest for Intelligence. “You wouldn’t want your whole visual system to become an elephant detector [just because you’ve learned to identify an elephant]. But this study went beyond that to say actually, when you learn ‘elephant,’ your IT does change a little bit to make it a little more relevant to elephants.”That likely has consequences for recognizing other visual features, too. Subtle changes in the IT cortex that support elephant recognition might also make you better at identifying things other than elephants, DiCarlo says. Likewise, the same changes might make it a little harder to identify something else.These kinds of consequences may be difficult to predict intuitively, but become obvious with computational modeling. For instance, the team’s models revealed that after learning to recognize new objects, the IT cortex contained more information about objects’ locations. By providing insights like these, models could aid the design of more effective training strategies for visual tasks, including for people with altered sensory processing, who may learn from visual information in atypical ways.