The Consciousness Network

See Also:
  <> Pennartz: The Brain's Representational Power - acedemic book, and the journal paper:
  <> Neurorepresentationalism, Cyriel M.A. Pennartz, What is neurorepresentationalism? From neural activity and predictive processing to multi-level representations and consciousness, Behavioural Brain Research, Volume 432,2022,113969, ISSN 0166-4328, https://doi.org/10.1016/j.bbr.2022.113969

This book introduced me to the term VTE,
He goes thru the key ideas of consciousness, a few that he developed himself:
Key idea #1: hierarchy from single-neurons, to ensembles (cortical columns), to unimodal networks, to multimodeal meta-networks
Key idea #2: consciousness is distributed through the brain

This book is good introduction to consciousness. It is also interesting if you have read other books on consciousness. It starts with the basics of perception and memory and how they evolved in earlier animals. It then leads into consciousness with one of the best definitions of it using the 'hallmarks of consciousness'. Then: "to give a definition of consciousness: the phenomenon by which we have a rich, multi-sensory overview of our situation in the world - how the world feels and looks, and how our body is positioned in it."

The author then works up to his model of consciousness called Neurorepresentationalism, which is one of the theories being investigated in the massive Templeton Foundation consciousness research granting program. It posits that consciousness is distributed over many brain regions, both lower levels (V1 for visual detail) and higher levels work together to allow us to understand what we see.

Along the way, Pennartz uses the insights of Descartes, Spinoza, Gödel, and a host of Neuroscientist to give each concept a deep context.

This book is written for a general, but scientifically curious, audience. It covers key ideas for consciousness such as memory, predictive coding, networks, brain regions, emotions, and basic functioning of the cortex and other key organs in the brain. The author is able to articulate complex concepts in an understandable way without dumbing them down.

Pennartz makes the case for multi-level mental models as the basis for our perception of the world. He is convincing in this. Quote:

p.131 "The information it gets delivered consists of nothing more or less than electrical pulses - spikes - and is thus encoded in cuneiform script. There is nothing or no one to provide your brain with a key to decipher this code before it enters your brain. Our brains are faced with the almost impossible task of making sense of all the inputs they receive - of interpreting the objects and events that the inputs cause. It is no exaggeration to say that we ourselves live in a brain-created simulation: a simulacrum. This is not a simulation that grants us a means to play 3D video games, but to mimic or model as closely as possible what is going on around us. To make complex decisions, we need a reliable overview of our situation and our body, an update of our sensory modalities in the here and now."

Pennartz is strongly in the camp that the more we learn about the brain, the easier the 'Hard Problem' becomes. Quote:

p.228 "... there is no 'I' that looks down on the lower levels from above, as it were. The 'I' or 'self' is not a thing but a complex construct that, as we saw earlier, consists of several dimensions, such as the 'I' as a body, as an acting entity that makes a difference in the physical and social world and has its own history, recorded in our autobiographical memory. Nor is there any homunculus at the higher level that takes in all the sensory inputs and welds them together into a unified experience. There is no separate observer who perceives things on lower levels. The network itself is both observer and producer of all that we perceive. If you put yourself in the shoes of a cinema visitor, you are not only the one who experiences the film, but also the film-maker; the distinction between making and seeing the film disappears."

Table 2.1 Cognitive abilities

p.113 "All well and good: consciousness offers us a richly varied overview of yourself and your world. ... we experience a scene consciously by being in the middle of it, in three dimensions plus time. Consciousness immerses us in the scene we are witnessing. It's not that we 'watch' this scene as if we were playing a video game. What we are experiencing is an immersive experience of the situation. We are part of a very realistic 'video game' that unfolds around us in three dimensions, enriched with all kinds of qualities other than purely visual ones. This immersion is something that imperceptibly forces itself on us when we wake up in the morning: we not only see something, but we also are somewhere. We are in a spatial situation. We have a body lying horizontally in a warm bed, in a bedroom where the morning light enters through the window. Sleepily we look around us and gauge what is going on. All this offers a sufficient foothold, after our previous detours, to give a definition of consciousness: the phenomenon by which we have a rich, multisensory overview of our situation in the world - how the world feels and looks, and how our body is positioned in it."

p.164 "The 'I' is about personal identity: who are we, really? ... you can of course refer to a body quite literally, but in a broader sense it is much more interesting to know who Matt is as a person: where he comes from, what family and friends he has, what he does for a living and what his character is like. Whenever Matt gets the chance to tell his story, he will heavily rely on his episodic memory: his personal, autobiographical memories, which he is able to put into words thanks to his general knowledge of facts and vocabulary. As Henry Molaison showed in Chapter 3, episodic memory strongly depends on the hippocampus and surrounding areas in the temporal lobe. This system gives us a 'historical self, endowing the larger system of self-representation a deep temporal dimension not found in raw sensations. Another component of our identity is 'personality'. This too is dependent on past life experiences, but relates more directly to traits such as temperament, extroversion or introversion, impulsivity or thoughtfulness - traits that are generally more strongly associated with the prefrontal cortex and basal ganglia than with the hippocampus."

p.165 "When you meet someone new at school or in your working environment, your brain consciously or unconsciously tries to build a model of his character and intentions - a mini theory of how the mind of this individual works: an estimate of what it is like to be this other person (also referred to as 'Theory of Mind'). This may result in empathy, other kinds of social sensitivity and morality, but during early development this process primarily revolves around attributing wishes, intentions and knowledge to other beings that may differ from your own."

p.166 "Spatial position, posture and movement: they provide essential information for constructing our conscious experience, and this experience helps us choose what we can take or leave given our position in the world. If we bring the 'self' into this world view, it is first of all a physical thing that we introduce, composed of different modalities. The simulacrum that our brains produce about how our bodies relate to the world is based on a subtle, subcutaneous conspiracy between sight, touch, internal bodily sensations and sense of balance (with hearing, taste and smell also occasionally taking part in the conspiracy). What we see is rarely the only content of what we are aware of - seeing is contained within a richer situation in which our body, with all its other senses, assumes a leading role, although we are not always attentive to it."

p.166 "The 'self', in short, can be understood as a higher-order construct, which is better left aside when we talk about primary conscious sensations. To see the colour red, it is not necessary for there to be a separate 'I' that perceives it. The 'I' as body or personality only appears much later in the circus programme of the mind. Returning to the idea of sensory inference, it may feel awkward that there is no one around in this process to do the watching, or experience representations of the outside world. There is, however, active brain tissue that does this. Helmholtz made a big step forward with his idea that the brain draws inferences from its sensory inputs, but also insisted that our conscious sensing must depend on a separate, psychological activity. We have now come far enough to take the next radical step. When it comes to conscious experience, this 'psychic' activity is the same as the brain activity generating a representation of the situation. No 'psyche' or 'I' is needed to get going separately with these representations, to 'make' them conscious or to take the next step in interpretation. The producer of representations is observer at the same time. This does not diminish the fact that the 'self' is an important construct, which we also need in our language to express who we are talking about. Within the spectrum of cognitive functions, language forms a separate category that strongly coheres with the functioning of our left-brain hemisphere. But this does not alter the fact that we do not need an 'I' or language for our basic sensory awareness."

VTE, vicarious trial and error

p.187 "Tolman's ideas spurred neurophysiology on to a quest that has yielded such brilliant discoveries as replay, the forward sweep and the encoding of a cognitive map in the hippocampus. His work on mental simulation of fictional action scenarios drew attention to a crucial component of the evolutionary success of homo sapiens: the internal formation of hypotheses and plans that reach far into the future. Tolman's work is also relevant to understand the neural basis of consciousness: I have previously argued that the function of consciousness - as a multisensory overview of one's situation in the here and now - is to support complex, goal-oriented behaviour, for which planning and internal deliberation are essential. Our consciousness is essential for feeding this internal jousting of deliberations with perceptual world updates.
...The second protagonist worthy of our admiration and praise is the rat."

p.186 "... all the elements of the system - molecules, synapses, neurons, networks - obey the laws of nature but, on the other hand, all the feedback loops and interactions within the system lead to a consistent and persistent 'will that remains unpredictable by outside observation. In the field of neural networks we speak of an 'attractor', a state to which the system as a whole moves. This is the state in which the system will remain stable after the initial phase of chaotic back-and-forth flicking has passed. Our brains, in short, are not 'free' or independent of the material elements of which they are composed, but they do have free will if we see this as a consistent result of their interactions that cannot be predicted from the individual behaviour of those elements."

p.192 "Recent research indicates that the human brain contains core areas required for different types of imagination (visual, auditory, somatosensory), and this core system is in close contact with the parietal lobe and the lower sensory areas for specific sensory modalities, such as the visual cortex? If the lower visual areas are lost through damage, the core imagination system can continue to communicate with the higher visual areas of the parietal lobe to maintain the generation of visual imagery. The loss of the lower areas will not lead to the detailed perceptions"

p.198 "At any given moment, your brain state is represented by the position of a golf ball rolling through the landscape. Shortly after the ball is hit into the field, it bounces a lot; this corresponds to a rapid moving back and forth between states. Later on, it rolls in a smooth motion: the states now alternate more gradually. The hills in the landscape are states that do not correspond to a structured experience. The ball will usually not come to a halt on top of a hill, but will move towards a valley - a structured representation formed by past experiences. Our understanding of this process has been strengthened by describing it physically with the concepts of a 'basin of attraction' (the valley) and 'attractor' (the end point where the little ball comes to rest). [jch - see 'attractor' in chaos theory] ... one state emerges as the 'winner'. Computer models indicate that this winner consists of a network of cells that are strongly interconnected and that happened to be strongly activated by the electrical stimulus - to out-compete rival groups of cells by way of inhibition."

p.223 "The Hopfield network shows us how the brain can process and store information in a dispersed, distributed way. The model puts flesh on the bones of the long-held suspicion that memory traces are not fixed in one microscopic place in the brain: there is no single cell that stores the memory of your grand-mother. A memory trace is spread across a synaptic network, and this network can lie within one area, such as the hippocampus, but depending on the complexity of the memory it can even be distributed across several areas. Together with other modellers and experimenters, Hopfield gave a strong impetus in the 1980s to the development of smart, self-learning computer models, which built on the work of McCulloch and Pitts and formed the basis of the current boom in artificial intelligence."

p.224 "A single neuron can only be active or inactive; its state is totally controlled by the input it receives from other cells. But the collective of neurons is capable of storing entire patterns and, when stimulated at random, exhibits a self-organizing dynamic in which the network sometimes converges to one stored pattern, and then rearranges itself to arrive at another pattern. It is a spectacle reminiscent of a flock of migratory birds that, with its own self-directed dynamics, draws three-dimensional forms against an autumn sky. In the landscape of all possible states, the current state of the network is represented by the position of the golf ball. Sometimes it keeps on rolling: the network continues to evolve towards new patterns. With a shard of information, the network manages to recall a stored pattern and make it complete: this is an emergent process because it cannot be achieved with single cells."

p.228 "Through its collective activity, the network transcends the level of numerical processing of information. After all, a hypothesis need not only concern num-bers, but can be about anything: shape, colour, temperature, texture, smell, taste, etc. The construction of a hypothesis is not something that follows in time after neuronal activity has taken place at the lower level: its realization consists of those patterns of neural activity. So the link between the hypothesis (at a high conceptual level) and neuronal activity (at a low conceptual level) is not 'causal' in the sense of having causal interactions like colliding billiard balls. It is more accurate to call it otherwise: an emergent correspondence between levels. Processes at lower levels correspond to processes at higher levels, albeit that the higher processes have new properties that are not found at lower levels."

p.228 "... there is no 'I' that looks down on the lower levels from above, as it were. The 'I' or 'self' is not a thing but a complex construct that, as we saw earlier, consists of several dimensions, such as the 'I' as a body, as an acting entity that makes a difference in the physical and social world and has its own history, recorded in our autobiographical memory. Nor is there any homunculus at the higher level that takes in all the sensory inputs and welds them together into a unified experience. There is no separate observer who perceives things on lower levels. The network itself is both observer and producer of all that we perceive. If you put yourself in the shoes of a cinema visitor, you are not only the one who experiences the film, but also the film-maker; the distinction between making and seeing the film disappears.

Note on Mental Models

On Oct 31, 2024, at 05:27, Cyriel Pennartz UvA wrote:
yes, a multi-level representation can be conceived of as a mental model. Imo, 'mental' can mean several things, but in this specific context it means a model we are conscious of, as the best 'superinference' (or prediction) of what our situation in our surroundings and our bodies is like. The brain generates all sorts of predictions, e.g. about the direct consequences of a motor action (cerebellar), and by far not all of these predictions result in conscious experience. What makes conscious representations special, as opposed to other representations, is that they are spatially encompassing and rich in qualities/modalities (hearing, vision, etc.) and submodalities (for vision: color, motion, shape etc.). Much of my book is targeting a better understanding of how the brain builds these world models from the bottom up, that is, from neurons to small ensembles, up to large networks that eventually span multiple modalities.

What is Neurorepresentationalism?

To the RIGHT: Fig. 3. Functional organization of different levels of representation in the construction of conscious experience.

5. Multi-level representations and the hard problem

"More concretely, multi-level representations underlying conscious experience have been proposed to be constructed bottom-up by the level of:
(i) single neurons, having the capacity to respond to single features;
(ii) ensembles of neurons, forming small, within-area local networks capable of pattern coding within a single submodality (e.g. shape);
(iii) unimodal metanetworks, which combine the hypotheses from lower-order ensembles into representations of objects considered within a single modality (e.g. all visual features making up a visual object) and
(iv) multimodal metanetworks, integrating the information coded by unimodal metanetworks into multisensory object representations"

Page 8:
"Neurorepresentationalism deploys its multi-level concept such that both low (e.g., V1) and high (e.g., inferotemporal) cortical areas can contribute to visual consciousness. For instance, inferotemporal areas of the non-human primate brain, containing neurons with very wide receptive fields, likely contribute position-invariant shape information to a perceptual representation, whereas V1 neu- rons, having small receptive fields, may contribute local visual details at a high resolution."

Table 1 Neurorepresentationalism considers five properties of phenomenal experience in healthy individuals to constitute inalienable hallmarks of consciousness.

Hallmark of conscious experience	Explanation
Multimodal richness	Conscious experience is qualitative in nature, i.e., it is characterized by sensations in multiple distinct modalities (vision, audition, somatosensation, smell, taste, vestibular sense). These main modalities can be partitioned into submodalities (e.g., for vision: color, texture, shape, motion, etc.).
Situatedness and immersion	In a conscious state we find ourselves situated in a space that is usually characterized by certain objects in the foreground and other stimuli in the background. Our body is experienced as immersed in the situation, occupying a central position relative to the surroundings.
Unity and integration	Consciousness is not made up of different elemental experiences, but is unified or integrated in that we have only one single experience at any given time. Our senses work together to enable the construction of an undivided, multimodal, spatially encompassing representation.
Dynamics and stability	Conscious experience is continuously updated following changes in the external environment and our body. Despite this dynamic aspect and ubiquitous movement of the head, eyes and other body parts, stationary objects in the environment are experienced as stable.
Intentionality	The property that a carrier substrate of consciousness can generate signals that are interpreted as, and refer to, something other than itself ('aboutness'). The brain's ability to interpret its own neural activity patterns not only pertains to ambiguous stimuli, illusions or hallucinations, but is considered a general and fundamental hallmark of consciousness.

Definition of consciousness - page 5 " a definition of conscious experience ..: it is the multimodally rich, dynamic survey of the subject’s current situation, including his own body and functionally earmarked for planned behavioral and cognitive actions in the future."

Page 9:
"Phenomenal experience just ‘is’."

p.41 "The [choice of odds] offers a glimpse into a component of emotions that is usually lurking silently in the background in the face of intense stimuli that trigger an emotion. This component consists of autonomous, bodily responses to sensory stimuli, such as an increased perspiration at the sight of a random playing card. To measure the intensity of the sweat response, the skin's conductivity to electrical current was recorded in prefrontal patients and healthy subjects.
In healthy people, perspiration increased when they saw a deck of cards that was covered but considered risky. The body signaled that some risky event was about to happen. With prefrontal lesions, this effect did not occur: the patient's body reacted as if nothing was at stake. An unconscious learning process - learning to assess the value of an object, in this case a deck of cards - was linked to an emotional response that is normally hidden from view - the secretion of sweat."

p.125 "the way that Johannes Muller proposed - and which is widely accepted today - is called 'labelled lines coding', Here, the sensory modality is determined by which sensory cell (and corresponding nerve fibre) is stimulated, while the pattern of pulses is not essential, The lines that run from the sensory cells to the brain are, as it were, labelled to indicate what kind of sensation you will experience, given the stimulation of your sense organs."