How do our brains create visual consciousness? Does seeing an apple require your brain to reprocess the image — or is the initial perception enough?
What gives rise to human cognition and subjective experience? And how does our brain create conscious experiences, particularly when it comes to vision?
While scientists generally agree cognition emerges from an incredibly complex network of neurons in our brains, with different regions working together to process information, form memories, regulate emotions and enable conscious thought, consciousness is a subject of ongoing debate between a few divergent theories.
Testing Predictions of First-Order and Higher-Order Theories of Consciousness – from a series of Structured Adversarial Collaborations funded by the Templeton World Charity Foundation – is bringing together a group of scientists who support one or the other of these opposing views to test three specific theories: Two higher-order theories, including the Higher Order Representation of Representation theory (HOROR) and the Perceptual Reality Monitoring theory (PRM); and a leading first-order theory, the Recurrent Processing Theory (RPT).
“In our project, the first order theory is championed by Victor Lamme,” says Dr. Biyu Jade He, Project Director and Associate Professor of Neurology, Neuroscience and Physiology, and Radiology at New York University Grossman School of Medicine, whose laboratory uses a combination of multimodal human brain imaging, brain stimulation, and computational approaches to investigate the neural mechanisms of perceptual processing in the human brain.
“First-order theories in general propose that for conscious perception, you have the neural representation in sensory areas of the brain and that, by itself, is sufficient to generate awareness. [For this project], we’ll test a particular first-order theory called the recurrent processing theory, which proposes that it's feedforward and feedback recurrent processing within the sensory hierarchy that generates awareness.”
Put another way, first-order theories suggest that conscious perception of, for example, motion, is directly determined by the parts of the brain that analyze motion. Conscious perception of a face is directly determined by the parts of the brain that analyze faces.
“In contrast, higher-order theory proposes that the sensory representations have to be re-represented in higher-order cortex to support awareness,” says Dr. He. “So, in our project, we’ll test two particular versions of higher-order theory. One is Perceptual Reality Monitoring theory from Hakwan Lau, and the other is the Higher-Order Representation of a Representation theory by Richard Brown.”
So, higher-order theories suggest that after your brain processes the image of the apple, another part of your brain – specifically the prefrontal cortex – needs to "check in" on that processing for you to become consciously aware of seeing the apple.
What makes this project particularly interesting is not just that the higher order theories will be pitted against the first order theory rather than against each other, but that groups of scientists have collaborated on designs of experiments that may challenge their own theories.
The first experiment – run by Assistant Professor of Psychological and Brain Sciences Dr. Rachel Denison at her lab at Boston University and Associate Professor of Cognitive Neuroscience Dr. Megan Peters at the University of California, Irvine – tests whether neural activity in the visual cortex, after you have equalized for performance-related compounds, directly correlates with visual consciousness.
“It tests the first-order theory more directly because first order theory's predictions are specifically tied to the visual cortical representation,” says Dr. He.
The first experiment examines whether there is “subjective inflation” in which perceptual experience in the periphery of vision is stronger than would be predicted by first-order theories. So, if subjective inflation is shown, it could challenge first-order theories. If not, one of the bases of higher-order theories would be challenged.
“The idea is that in peripheral vision we see things more colorfully than would be predicted by first-order representations of color,” says project co-lead Dr. Ned Block, Silver Professor of Philosophy, Psychology, and Neural Science at New York University.
“The higher-order people think the fact that we see all that color in our peripheral vision must show that peripheral color is contributed by higher-order areas. So, one thing we needed to do was to measure whether there really is peripheral inflation (a level of detail or clarity in the peripheral visual field, that goes beyond first-order representation) by having very detailed experiments that looked in detail as to whether it really is true. And, as we are showing, it is true. There is peripheral inflation.”
The second experiment – led by Dr. He (at her NYU lab) and Dr. Jan Brascamp, Associate Professor of Cognition and Cognitive Neuroscience at Michigan State University – tests the higher-order theories as they pertain to the prefrontal higher-order neural representation underlying conscious perception by investigating whether after the post-perceptual functions are controlled for (e.g., processing after conscious perception has happened) prefrontal cortical activity directly correlates with conscious visual perception.
Specifically, the second experiment looks at "change blindness," moments when people fail to notice obvious changes in what they are looking at. Essentially, this experiment aims to separate conscious experience from activity in the prefrontal cortex – success would challenge higher-order theories, while failure would challenge first-order theories.
“The idea is people see these changing pictures and very often do not notice the changes,” says Dr. Block. “One theory of why they don't notice the changes is they have very sparse perception, and don't see the features that change. But another theory is they don't conceptualize those properties. [In other words], they do consciously see the features that change but they don't conceptualize them at a level that will allow them to make comparisons.
“The higher-order view is associated with the sparse perception idea because it correlates to there being limits on cognitive representation.”
Both experiments, which involve healthy volunteers viewing different visual displays and responding to them, are being replicated concurrently in multiple labs by teams of experts (four neuroscientists and two philosophers), and are being conducted both behaviorally, as well as with brain activity measurements using MRI scanners.
“We’ve published some smaller papers with relatively unexpected behavioral findings,” says Dr. He, referring to two not theory-adjudicating papers published earlier this year. One about methods in the Neuroscience of Consciousness (“A novel, semi-automatic procedure for generating slow change blindness stimuli,” https://academic.oup.com/nc/article/2024/1/niae004/7606609), and the other about results in the Journal of Vision (“Memory representations during slow change blindness,” https://jov.arvojournals.org/article.aspx?articleid=2800790).
“Though the findings were not conclusive for testing the hypothesis or the theories, they were nonetheless very interesting, very intriguing, and told us something new about change blindness, a visual phenomenon that people have been intrigued about for a long time,” says Dr. He.
“And for experiment one, we have some initial behavior results that are highly interesting and weakly supporting the higher-order theory. But it's not totally conclusive for hypothesis testing either.” The initial result was reported at two conferences earlier this year, according to Dr. He, and is currently being written up in article format.
“The most conclusive evidence for the experiments is expected to come from the imaging component, which is still ongoing,” says Dr. He, who hopes both experiments will be completed in 2025.
While neither experiment is likely to completely solve the mystery of consciousness, they represent important steps forward in understanding one of the most fundamental aspects of human experience - how we become aware of the world around us.
“I think regardless of what we find in terms of theoretical adjudication, these are two really cool experiments from a cognitive neuroscience perspective, in a way that probes some of the hard issues in the field of consciousness, especially conscious perception,” says Dr. He. “I think we're going to have really interesting data, large data sets reproduced across labs with innovative paradigms that will shed light onto some of the longstanding questions in the field of conscious perception, and the neural basis of conscious perception.”
To better understand existing theories of consciousness, Templeton World Charity Foundation (TWCF) funds a series of 5 ambitious research projects, known as Structured Adversarial Collaborations, to allow researchers to evaluate competing hypotheses through carefully designed experiments and rigorous cross-laboratory validation.
This blog is part of a series that complements the launch of Accelerating Research, a unique platform developed by Dawid Potgieter and DataCite with TWCF funding, showcasing results from each Structured Adversarial Collaboration project as it progresses.