2.1 Introduction

From the moment we wake until we fall into a dreamless sleep, our brain constructs experiences ranging from the vibrant hues of a sunrise to the surreal narratives of dreams. Defining and measuring consciousness is fraught with difficulties. Indeed, consciousness has been described as a slippery term (Blackmore, Reference Blackmore2005), referring to various phenomena such as self-awareness, reflective thought, and sensory perception. Another difficulty is that, unlike other phenomena in the sciences, consciousness is not publicly observable, making it challenging to study objectively. A frequently cited definition in the literature which seems to intuitively capture what consciousness is comes from Thomas Nagel’s seminal paper ‘What is it like to be a bat?’ (Nagel, 1974). According to Nagel, a system is conscious if there is something it is like to be that system – for example, if it experiences temperature, colours, sounds, or smells. Presumably, there is nothing it is like to be a thermostat or a mobile phone; however, there is something it is like to be a human or another animal, such as a bat. In the case of humans, there is subjective experience: the blinding white of fresh snow, the tanginess of a lemon, the warmth of sunshine on skin, and the pain of a bee sting. In this Element, when we discuss consciousness, experience, or awareness, we are referring specifically to visual experience: awareness of colours, objects, motion, faces, and scenes.

2.2 Phenomenal and Access Consciousness

A distinction is often made between phenomenal and access consciousness (Block, Reference Block1995; Block, Reference Block2007). Phenomenal consciousness refers to subjective qualities, such as the vivid redness of a sunset or the itchiness of a wool sweater.In contrast, access consciousness refers to the computational, functional, neurological, and behavioural mechanisms that underlie how we report, make decisions, remember, and use information about experiences. For example, recognizing and reporting that we are seeing a red sunset and deciding to alleviate an itch involves access consciousness.

Whether this distinction can be made remains a prominent issue in consciousness research (Kouider et al., Reference Kouider, de Gardelle, Sackur and Dupoux2010). The argument centres on whether we are conscious of more information than we have cognitive access to. One perspective is that phenomenal consciousness ‘overflows’ access consciousness (Block, Reference Block2011). In other words, observers have a richer phenomenology – the subjective experience of sensory qualities – than can be accessed by cognitive processes such as attention, working memory, and reporting mechanisms (Amir et al., Reference Amir, Assaf, Yovel and Mudrik2023; Block, Reference Block1995; Block, Reference Block2005; Lamme, Reference Lamme2004).

Conversely, others argue against the overflow hypothesis, challenging the distinction between phenomenal and access consciousness (Cohen & Dennett, Reference Cohen and Dennett2011; Dehaene et al., Reference Dehaene, Changeux, Naccache, Sackur and Sergent2006; Kouider et al., 2010; Naccache, Reference Naccache2018; Overgaard, Reference Overgaard2018). According to these researchers, the functions of consciousness (access consciousness) exhaust its nature. There is no consciousness without function. Under such accounts, an overflow of phenomenal consciousness is an illusion (Kouider et al., 2017).

2.3 Levels and Contents of Consciousness

Another important and contested distinction is between global states (also called levels) and local states or the contents of consciousness (Bayne et al., Reference Bayne, Hohwy and Owen2016; Hohwy, Reference Hohwy2009; Koch et al., Reference Koch, Massimini, Boly and Tononi2016). Global states refer to varying states of awareness, ranging from full alertness to unconsciousness. These include wakefulness, sleep, dreaming, anaesthesia, coma, vegetative state, and locked-in syndrome, each characterized by distinct neural activity and behavioural patterns (Laureys, Reference Laureys2005; Laureys et al., Reference Laureys, Gosseries and Tononi2015). The contents of consciousness encompass the objects, sounds, smells, and scenes represented in experience at any given moment. These include sensory perceptions (seeing a cat, hearing it meow), emotions (feeling happy to see the cat), thoughts (thinking about preparing food for the cat), memories (recalling seeing the cat for the first time), hallucinations (seeing the cat when it is not there), and the contents of imagination (seeing the cat flying with wings).

Research suggests that global states and contents of consciousness are interrelated (Aru et al., Reference Aru, Suzuki, Rutiku, Larkum and Bachmann2019). Conscious content depends on the global state. For example, degraded levels of consciousness, such as those experienced in minimally conscious states (MCS), often reduce the vividness and clarity of conscious contents (Overgaard & Overgaard, Reference Overgaard and Overgaard2010). Different global states might gate or limit the range of conscious contents, affecting the quality and extent of what is experienced. This gating mechanism can result in varying degrees of richness in conscious experience across different states, revealing an interdependence between local (content) and global states (Bayne et al., Reference Bayne, Hohwy and Owen2016).

2.4 Measuring the Contents of Consciousness

In the laboratory, experimentalists treat consciousness as a variable and create situations where participants are either aware or unaware of a stimulus. Known as the contrastive method, this approach allows researchers to compare conscious and unconscious processing. By creating conditions where participants are aware of the stimulus and others where they are not, researchers can isolate the neural and cognitive mechanisms underlying conscious perception from those underlying unconscious processing (Baars, 1993). Here, some of the standard ways of achieving this are briefly described (for a more exhaustive review, see Kim & Blake (Reference Kim and Blake2005), Bachmann et al. (Reference Bachmann, Breitmeyer and Öğmen2011), and Bachmann & Aru (Reference Bachmann and Aru2023)). We will refer to some of these techniques throughout this Element.

Masking: A mask is another visual stimulus that is presented either before or after the target stimulus to interfere with its processing. By manipulating the interval between the target and the mask, researchers can effectively ‘mask’ the target, studying the conditions under which it is consciously and not consciously perceived (Breitmeyer, Reference Breitmeyer2007; Breitmeyer & Ogmen, Reference Breitmeyer and Öğmen2006). This allows for the differentiation between neural activity associated with unconscious and conscious processing. Different kinds of masking techniques are used to achieve this effect. Backward masking involves presenting the mask following the target stimulus, which interferes with the processing of the target and prevents it from reaching conscious awareness. Forward masking involves presenting the mask before the target stimulus, affecting the processing of the subsequent target. In metacontrast masking, the mask does not spatially overlap the target but closely follows it in time, while surrounding it spatially, reducing target visibility. Finally, sandwich masking involves presenting the target stimulus between two masks, one before and one after. These variations of masking help researchers dissect the temporal dynamics of visual processing and the conditions under which stimuli are consciously and unconsciously perceived (Breitmeyer, Reference Breitmeyer2007; Breitmeyer, Reference Breitmeyer2014). As well as measuring the threshold for conscious perception, many masking studies employ accuracy or sensitivity as the dependent measure.

Binocular Rivalry: In this method, each eye is shown a different image: a house to the left eye and a face to the right eye, for instance. Instead of perceiving an amalgamation of house and face, perception vacillates: sometimes, the participant reports seeing a face and sometimes a house. Using this method, experimentalists can measure the processes underlying conscious perception by comparing the activity when the participants are conscious of the stimulus to when they are not while keeping the sensory input constant. This contrast allows researchers to differentiate the neural activity associated with conscious perception (when one image is dominant) from that associated with unconscious processing (when the image is suppressed). For example, studies show that neural activity in the visual, parietal, and prefrontal cortex correlates with alternating perceptions during binocular rivalry (Blake & Logothetis, Reference Blake and Logothetis2002). Through such research, researchers can identify the neural correlates of visual conscious perception (Logothetis et al., Reference Logothetis, Leopold and Sheinberg1996; Sterzer et al., Reference Sterzer, Kleinschmidt and Rees2009; Tong et al., Reference Tong, Meng and Blake2006).

Inattentional Blindness: Inattentional blindness (IB) occurs when an observer is unaware of a visible stimulus because their attention is engaged elsewhere (Mack & Rock, Reference Mack and Rock1998; Most et al., Reference Most, Simons, Scholl, Jimenez, Clifford and Chabris2001; Most et al., Reference Most, Chun, Widders and Zald2005). Paradigms of IB reveal the limits of attention and conscious perception and illustrate how unattended stimuli can evade awareness. A classic example is a study by Simons and Chabris (Reference Simons and Chabris1999), where participants watched a video of people passing a basketball and failed to notice a person in a gorilla suit walking through the scene, a striking example of the limitation of conscious perception. Inattentional blindness provides researchers with a tool for investigating the selective nature of attention and the mechanisms by which certain stimuli fail to enter consciousness (Jensen et al., Reference Jensen, Yao, Street and Simons2011).

Visual Illusions: Visual illusions are an important tool in studying consciousness as they reveal the rules and logic of the construction of percepts. They occur when there is a discrepancy between the external reality and our experience of it and thus give insight into how the brain interprets sensory information to create a perception. Motion-induced blindness presents one such tool. In this illusion, a salient constantly present stimulus in the visual field perceptually disappears and appears when surrounded by a moving pattern. By contrasting neural activity when the static stimulus is visible (conscious processing) and when it disappears (unconscious processing), researchers can reveal the cognitive and neural processes associated with conscious perception (e.g., Bonneh, Cooperman, and Sagi (Reference Bonneh, Cooperman and Sagi2001)).

Continuous Flash Suppression (CFS): By rapidly presenting a series of images to one eye while showing a static image to the other, researchers can suppress the perception of the static image for several seconds. For example, in one CFS study, Jiang, Costello, and He (Reference Jiang, Costello and He2007) demonstrated that emotional faces break through suppression faster than neutral faces, indicating that certain stimuli have privileged access to consciousness due to their emotional salience. Other CFS research reveals that high-level visual processes can occur without conscious perception. For example, in an fMRI study, Sterzer et al. (Reference Sterzer, Haynes and Rees2008) found that the category of objects invisible to participants due to CFS was still encoded in high-level visual areas, such as the fusiform face area (FFA) and the parahippocampal place area (PPA), indicating that such processing can occur without conscious perception.

Attentional Blink (AB): Attentional blink is a phenomenon where an observer fails to detect a second stimulus presented within 200–500 milliseconds after the first stimulus. This occurs because processing the first stimulus occupies attentional resources, creating a temporary ‘blink’ during which the second stimulus is missed (Raymond, Shapiro, & Arnell, Reference Raymond, Shapiro and Arnell1992). By comparing neural activity during the blink period (unconscious processing) to when the second stimulus is detected (conscious processing), researchers can study the temporal dynamics of attention and its role in conscious perception. Researchers have found that the AB can reveal how attentional resources are allocated over time and how this affects awareness (Martens & Wyble, Reference Martens and Wyble2010).

Using these paradigms and many others, experimenters can measure thresholds for the conscious perception of different stimuli, determine when stimuli enter awareness, and explore the conditions under which stimuli are consciously perceived. Additionally, isolating the psychological and neural processes that occur when stimuli are consciously perceived versus when they are not allows researchers to uncover the neural correlates of consciousness (NCC), the minimal neuronal mechanisms jointly sufficient for a specific conscious experience (Dehaene & Changeux, Reference Dehaene and Changeux2011; Koch et al., Reference Koch, Massimini, Boly and Tononi2016).

In the next section, we will explore the cognitive and neural underpinnings of perception and how the brain’s predictive and inferential mechanisms create perceptual reality. With this foundational understanding, we will go on to examine the roles of attention, expectation, and their interplay in conscious perception, building on the concepts introduced here and integrating them within a theoretical framework, which is introduced in the following section.

3.1 Introduction

Gazing upwards as we lie in a park on a sunny day, we perceive the colours, textures, and motion of the leaves against a blue sky. We do not see a flat scene (the two-dimensional image on the retina) but one with spatial and temporal depth: the leaves nearby may be perceived as within arm’s reach, while those higher in the tree are perceived as relatively further away. Encased in the skull, the brain’s point of contact with the world beyond is the relentless barrage of information from that world impinging on sensory receptors. In the case of vision, light is reflected from the three-dimensional world (the distal stimulus) and forms a two-dimensional image on the retina (the proximal stimulus). From this sensory input pattern, the brain faces the challenge of recovering the distal stimulus (the cause of the input) from the proximal stimulus (its effect on the sensory receptors) (Hohwy, Reference Hohwy2013; Marr, Reference Marr1982). This involves moving from two dimensions to three, which is not a well-defined function since each point in the 2D image could correspond to an infinite number of points in the 3D environment. An infinite number of three-dimensional objects of different sizes, shapes, slopes, and slants at various distances could create an identical two-dimensional image on the retina (Figure 1). Given this, how does the brain recover the world out there from the highly ambiguous input? How does it determine the cause from its effect on the retina when that effect has multiple possible causes?

Figure 1 The inverse problem: reconstructing three-dimensional space from two-dimensional retinal images.

This can be illustrated with bistable figures, two-dimensional stimuli alternately perceived as two different three-dimensional objects (Blake & Logothetis, Reference Blake and Logothetis2002), such as the Necker Cube (see Figure 2). Given on the retina is a two-dimensional shape, yet we perceive a three-dimensional cube. Moreover, continued observation changes our perception of the cube, revealing a new orientation. The perception of bistable figures demonstrates that the brain cannot settle on a single interpretation of the retinal image, revealing the inherent ambiguity in the visual input and the constructive nature of perception. When viewing bistable figures, we experience this in real time.

Figure 2 Necker Cube (left) demonstrating perceptual bistability. The centre and right cubes illustrate different perceptual interpretations, where the shaded surfaces represent possible front and rear faces of the cube, leading to alternative 3D perceptions depending on which face is seen as being in front.

Furthermore, visual illusions illustrate how what we perceive is not strictly determined by the pattern of stimulation in the retinal input. In the Kanizsa triangle, we perceive an illusory white triangle superimposed on three circles (see Figure 3). We can see the edges of the triangle, which appears to be whiter than its white background. However, if we take a photometer and measure the luminance profile of the ‘triangle’, we will see there is no change in luminance between what we perceive as the edges and the background. The triangle is a construction of your visual system.

Figure 3 Kanizsa triangle illustrating illusory contours.

This inverse problem of vision, recovering the causal structure of the three-dimensional external world from the two-dimensional sensory input, has been recognized historically (Berkeley, Reference Berkeley and Dennis1948), as has the constructive nature of experience (Kant, Reference Kant2001/Reference Kant1783; Swanson, Reference Swanson2016). Helmholtz introduced unconscious inference, a process of unconscious calculations to interpret highly ambiguous sensory input (Helmholtz, Reference Helmholtz1867/1925). He proposed the likelihood principle: perceptual systems favour the interpretation of sensory data most consistent with expectations. For example, suppose you see a vague shape moving in a forest. In that case, your brain is more likely to interpret it as an animal rather than an inanimate object because, based on past experiences, it is more probable that moving shapes in such contexts are living creatures. This probabilistic approach allows the brain to construct a coherent representation of the world, even when the sensory input is ambiguous or incomplete. As discussed later, Helmholtz’s unconscious inference and likelihood principle remain foundational in contemporary theories of perception (Clark, Reference Clark2013).

3.2 Bottom-up and Top-down Processing

Perception involves the interaction of bottom-up and top-down processes. Bottom-up processing refers to the sensory-driven aspects of perception, where information from the sensory receptors is relayed to the brain for further processing. This begins with sensory input, building a percept through a series of hierarchical stages – that is, from cells that encode light and dark in the retina to cells in early cortical areas that encode lines and edges, to cells in higher visual areas that encode faces and objects (Marr, Reference Marr1982). Perception is thus seen as a process of integrating sensory inputs through multiple levels of processing in a feed-forward direction: basic features such as colour, brightness, edges, and motion providing the building blocks for high-level representations such as faces, objects, and scenes.

In contrast, top-down processing involves cognitive factors such as expectations and knowledge based on prior experiences. Top-down processes can modify or shape sensory information, leading to perceptual interpretations that align with expectations and prior knowledge. This type of processing is conceptually driven and helps resolve ambiguities in sensory input by applying learned information and contextual clues (Bar, Reference Bar2004; Gregory, Reference Gregory1980; Kersten et al., Reference Kersten, Mamassian and Yuille2004).

A demonstration of the interrelation between bottom-up and top-down processing can be seen in Figure 4. Upon first seeing such a stimulus, observers do not see much, just the blotches of black and white that make up the stimulus However, if we keep looking, we perceive this array of black and white blotches as a Dalmatian dog, perhaps in dappled sunlight, sniffing its way along a path. Here, we see top-down processes – predictions about the world based on past experience – interacting with the bottom-up input, the ambiguous image on the retina, shaping what we perceive. Now, whenever you look at the image, you will most likely perceive it as the dog. Note that the image does not have a dog shape. Our brains infer it is there, and this inference determines what we perceive. This shows the powerful influence of top-down processing in interpreting and making sense of ambiguous sensory information. Prior knowledge and expectations can significantly shape our perceptual experience.

Figure 4 Perceptual illusion demonstrating a hidden Dalmatian dog.

3.3 Bottom-Up Theories of Perception

How much influence do bottom-up processes and top-down processes have on what we perceive? Bottom-up theories of perception emphasize the role of sensory input in shaping perceptual experience. Traditionally, this process has been considered to work roughly in the following way: sensory information impinges on the retina, which is the first step in a chain of (mostly) feed-forward information processing from the retina via the optic nerve to the back of the brain, the occipital cortex, and on to the temporal lobes. Perception is seen as a bottom-up process: from processing simple light/dark distinctions at the retinal level, to cells that are sensitive to edges and line orientation, colour, motion, and spatial frequencies, and at higher levels (i.e., those in the temporal lobe), cells that respond to faces, objects, and scenes. In seminal studies for which they won the Nobel Prize, Hubel and Wiesel demonstrated that cells in the retina of anaesthetized cats responded to spots of light located in very specific areas of the retina, while cells in the visual cortex responded to edges, or lines composed of these spots of light again located in very specific areas of the retina (e.g., Hubel & Wiesel, Reference Hubel and Wiesel1959). Later work revealed that as one moved further away from the retinal input in the visual system, cells responded to visual features, and became increasingly specialized in responding to faces, tools, scenes, and objects (e.g., Hubel & Wiesel, Reference Hubel and Wiesel1962; Kanwisher et al., Reference Kanwisher, McDermott and Chun1997; Grill-Spector & Malach, Reference Grill-Spector and Malach2004). Such a flow of information, in which simple light–dark distinctions lead to edge detection and object perception and recognition, from the visually simple to the more complex, suggests that perceptions are constructed from the sensory input in an ascending hierarchy of layers of information processing.

An influential bottom-up theory is David Marr’s computational approach to vision (Marr, Reference Marr1982), which describes vision as a series of information-processing stages that transform raw visual input into a detailed, three-dimensional representation. Marr’s model includes several key stages:

• The Primal Sketch: This stage captures basic features such as edges, textures, and simple geometric structures, representing these features in a way that highlights the intensity changes in the visual field.

• The 2.5D Sketch: This stage integrates information about depth and surface orientation, adding a sense of the spatial layout by incorporating depth cues like shading, texture gradients, and binocular disparity, providing a viewer-centred representation of the scene.

• The 3D Model Representation: This final stage integrates the information from the 2.5D sketch to form a complete, three-dimensional understanding of the visual scene, allowing for recognition and manipulation of objects regardless of the viewer’s perspective.

For example, consider a coffee cup on a table: at the primal sketch stage, the visual system detects the edges of the coffee cup, the handle, and the table, identifying the boundaries where there is a significant change in light intensity, outlining the shapes present in the scene. At the 2.5D sketch stage, depth cues are added to the visual representation, incorporating information about the curvature of the cup, the roundness of the handle, and the angle of the table surface. Finally, at the 3D model representation stage, the brain constructs a detailed three-dimensional model of the scene, recognizing the coffee cup as a distinct object, understanding its three-dimensional form, and allowing the viewer to recognize the cup, anticipate how it can be picked up, and understand its spatial relationship with other objects on the table. By processing visual information through these stages, Marr’s computational approach demonstrates how the brain transforms raw sensory input into a rich, three-dimensional representation.

This model underscores a predominantly bottom-up process in visual perception, where the interpretation of our visual world is assembled in a hierarchical fashion, layer by layer, culminating in the rich tapestry of the reality we perceive. But does this account for the flipping of perception when viewing the Necker Cube? Does this account for the ghostly edges one sees when looking at the Kanizsa triangle? Can this bottom-up explanation account for all experiences?

3.4 Top-Down Theories of Perception

According to perhaps the most widely accepted theory of vision, perception is a construction created mainly by top-down hypothesis-generating mechanisms, whose predictions are based on expectations (Clark, Reference Clark2013; Gregory, Reference Gregory1980;Hohwy, Reference Hohwy2013; Hohwy, Reference Hohwy2020; Seth, Reference Seth2021). Expectations are foundational components of internal models which predict the cause of sensory input (Series & Seitz, Reference Seriès and Seitz2013; Summerfield & Egner, Reference Summerfield and Egner2009). According to such accounts, the nervous system is essentially a prediction machine that attempts to minimize surprise (errors between the predictions and the sensory information, i.e., expectation violations) and maximize expectation (Friston et al., Reference Friston, Adams, Perrinet and Breakspear2012) using a form of Bayesian inference (Knill & Pouget, Reference Knill and Pouget2004).

3.5 Bayesian Inference and Predictive Coding

Today, modern computational theories are beginning to provide a detailed account of how perception is constructed from prior knowledge and sensory input. While predictive processing encompasses the broad principle that the brain is continuously generating and updating predictions about sensory inputs, predictive coding (Mumford, Reference Mumford1992; Rao & Ballard,Reference Rao and Ballard1999) specifically involves a process of precision-weighted prediction error minimization through hierarchical Bayesian inference (Clark, Reference Clark2013; Friston, Reference Friston2005; Hohwy, Reference Hohwy2013). According to this framework, what we perceive results from this precision-weighted prediction error minimization process.

Bayes’ theorem is fundamental to this framework. Originating from the work of Thomas Bayes in the eighteenth century, it provides a mathematical foundation for updating beliefs about a hypothesis based on new evidence (Bayes, Reference Bayes1763; Jeffreys, Reference Jeffreys1998). Bayes’ theorem formalizes this process:

$P\text{(}H\mid E\text{)}\text{=}P\text{(}E\mid H\text{)}\times P\left(H\right)\text{/}P\left(E\right),$

where

To illustrate, consider this – you hear a scratching noise. What is it? This sound could originate from multiple sources (the inverse problem). Is it rain tapping on the window, a cat scratching at the door, or a tree branch scraping against the outside wall? Using Bayes’ theorem, your brain evaluates the likelihood of each source based on prior knowledge and current sensory input. If you live in a quiet area with no history of break-ins, the probability P(H) of a burglar being the source is low. Conversely, if you know trees often scrape against your wall when windy, this prior probability is higher. Combining these priors with the evidence likelihood P(E∣H), your brain updates its belief to identify the most probable noise source.

In predictive coding, approximate Bayesian inference is used due to the computational challenges of exact Bayesian inference (Friston, Reference Friston2010). According to this theory, the brain generates predictions about expected input at each hierarchical level, comparing these predictions to the input at the level below to calculate prediction errors (mismatches between the prediction and the sensory input). These errors are propagated up the hierarchy to refine future predictions (Clark, Reference Clark2013). This process minimizes prediction errors, enhancing perception and action efficiency. Lower levels process basic sensory features, while higher levels integrate these into detailed representations, generating predictions for the level below (Clark, Reference Clark2015; Friston, Reference Friston2005; Millidge et al., Reference Millidge, Seth and Buckley2022) (see Figure 5). The result is an informationally rich model of the world inside the brain.

Figure 5 Hierarchical processing of sensory input through prediction and prediction error across multiple layers.

Generative models, central to predictive coding, simulate expected sensory signals, encapsulating statistical regularities of sensory inputs. When sensory input mismatches predictions, prediction errors signal the need to update generative models. These errors are propagated up the hierarchy, adjusting higher-level predictions to better predict lower-level activity (Rao & Ballard, Reference Rao and Ballard1999).

Precision weighting adjusts the influence of prediction errors based on reliability. In statistical terms, precision is the inverse of variance, representing the confidence in the accuracy of sensory inputs. High-precision errors (reliable inputs) weigh more in updating models, while low-precision errors (noisy inputs) are downplayed. This allows for prioritization of important information (Yon & Frith, Reference Yon and Frith2021). For example, in dim light, the brain relies more on prior knowledge than unclear sensory evidence, demonstrating the adaptive nature of perception under varying conditions.

Predictive coding frames perception as an active process of hypothesis testing and error correction. Navigating your way through the busy city streets, your brain uses internal models from past experiences to predict the movement of people around you. Sensory input about their movements is continuously compared to these predictions, and discrepancies (prediction errors) are used to update the model. This hierarchical process involves basic motion predictions and interpretations of complex social behaviour, with precision weighting prioritizing reliable information (Friston, Reference Friston2010). Similarly, in a fast-paced environment like driving, your brain predicts the behaviour of other vehicles based on their speed and trajectory. As the situation evolves, these predictions are updated using incoming visual data. Precision weighting helps the brain prioritize the most reliable information, such as the immediate visual feedback of other cars’ positions, enabling swift and accurate reactions (Yon & Frith, Reference Yon and Frith2021).

Helmholtz’s theory of unconscious inference laid the foundation for understanding perception as an active inferential process. He proposed that the brain interprets sensory input by inferring the most likely causes based on prior knowledge and sensory data. Predictive coding builds on this idea, suggesting that the brain continuously generates and updates predictions about the environment, minimizing prediction errors through Bayesian inference. Thus, predictive coding can be seen as a modern extension of Helmholtz’s theories, integrating them with contemporary computational and neuroscientific approaches to explain how the brain constructs perceptual reality.

In the following section, we will explore how attention shapes conscious perception. We will then investigate the role of expectation in visual experience and the interaction between attention and expectation.

4.1 Attention and Perception

Attention has limited capacity, allowing us to attend to about three to four objects simultaneously (Cowan, Reference Cowan2001; Luck & Vogel, Reference Luck and Vogel1997). While detailed processing of individual objects is limited, larger ensembles or global patterns can be processed more extensively as summary statistics (Jackson-Nielsen et al., Reference Jackson-Nielsen, Cohen and Pitts2017). This trade-off shows that the number of objects attended to and the detail processed are inversely related. Perceptual load, the quantity and complexity of information we can attend to simultaneously, affects our ability to process unattended information (Lavie, Reference Lavie2005). Attention can be directed at spatial locations (Posner, Reference Posner1980), objects (Egly, Driver, & Rafal, Reference Egly, Driver and Rafal1994), or features like colour or shape (Saenz, Buracas, & Boynton, Reference Saenz, Buracas and Boynton2002). Attention enhances spatial and temporal resolution, contrast sensitivity, and feature binding, and reduces crowding (Carrasco & McElree, Reference Carrasco and McElree2001; Carrasco et al., Reference Carrasco, Williams and Yeshurun2002; He, Cavanagh, & Intriligator, Reference He, Cavanagh and Intriligator1996; Treisman & Gelade, Reference Treisman and Gelade1980; Yeshurun & Carrasco, Reference Yeshurun and Carrasco1998). It also modulates sensory processing, enhancing neural activity related to attended stimuli while suppressing that of unattended stimuli (Desimone & Duncan, Reference Desimone and Duncan1995).

Attention can be overt (directly focusing sensory organs) or covert (mentally focusing without physical movement) (Posner, Reference Posner1980). Covert attention, such as spatially focused covert attention, involves focusing on a location without eye movements (Posner, Reference Posner1980).

Attention is also categorized as endogenous (internally driven) or exogenous (externally driven) (Corbetta & Shulman, Reference Corbetta and Shulman2002). Selective attention focuses on locations, objects, features, or tasks while ignoring others, as seen in listening to one conversation at a noisy party (Broadbent, Reference Broadbent1958), and can also be distributed across multiple elements simultaneously, like tracking players in a sports game (Neisser & Becklen, Reference Neisser and Becklen1975).

4.2 Neural Mechanisms of Attention

A theory of attention proposed by Desimone and Duncan (Reference Desimone and Duncan1995) has been highly influential in the cognitive sciences. According to this model, the flood of incoming visual data competes for limited neural representation. Attention enhances the processing of a stimulus by biasing the competition in favour of the attended stimulus. For example, studies show that attention increases visual responses in the extrastriate visual cortex (e.g., V4) by boosting the activity of neurons representing the attended stimulus while suppressing the activity of neurons representing competing stimuli (Luck et al., Reference Luck, Chelazzi, Hillyard and Desimone1997; Reynolds et al., 2000). This modulation has also been observed in other regions, including the prefrontal cortex and the lateral intraparietal area (Bisley & Goldberg, Reference Bisley and Goldberg2010; Buschman & Miller, Reference Buschman and Miller2007). The biased competition model integrates both top-down and bottom-up processing in selective attention. Top-down processes involve cognitive control from higher brain regions that bias the competition based on the individual’s goals and expectations. Bottom-up processes are driven by the properties of the stimulus, such as its salience and novelty.

In predictive coding, attention has been conceptualized as precision, the accuracy or reliability (more precisely, the inverse variance) of specific sensory signals in the context of prediction errors (Hohwy, Reference Hohwy2012; but see Aitchison & Lengyel (Reference Aitchison and Lengyel2017) for a differing perspective). According to this perspective, attention enhances the reliability of prediction errors by assigning greater weight to specific sensory inputs over others. By increasing the precision of relevant information, attention helps the brain minimize prediction errors more effectively, leading to a more accurate perception of the world (Feldman & Friston, Reference Friston2010). For example, Kok et al. (Reference Kok, Jehee and de Lange2012) conducted a study where they manipulated attention and prediction independently using an fMRI setup and found that attention boosts the precision of perceptual inference, enhancing the neural response to predicted stimuli when they are attended compared to when they are unattended. This interaction supports the view that attention increases the synaptic gain of neurons representing sensory data (prediction errors), thus optimizing perceptual accuracy (Feldman & Friston, Reference Friston2010; Kok et al., 2011). We will explore this in more detail in later sections.

4.3 The Debate over the Necessity of Attention for Conscious Perception

As the Introduction outlines, a debate persists in the cognitive sciences regarding the relationship between attention and conscious perception (Block, Reference Block2011; Lamme, Reference Lamme2010; Koch & Tsuchiya, Reference Koch and Tsuchiya2007). In this section, we will examine evidence from various experimental paradigms to explore whether attention is necessary for conscious perception. We begin by discussing findings from studies on inattentional blindness (IB), change blindness (CB), the attentional blink (AB), and ERP research suggesting that attention is necessary for conscious perception. Next, we present counterarguments based on research indicating that consciousness can occur without attention, focusing on summary statistics, gist perception, ERPs, and iconic memory (IM). Finally, we critically evaluate evidence from both sides and argue that the weight of evidence suggests that attention is necessary for conscious perception.

5.1 Expectation and Perception

Expectations are predictions that the brain generates based on prior knowledge, contextual cues and past experiences about the noisy, ambiguous sensory input (Kok et al., Reference Kok, Brouwer, van Gerven and de Lange2013; Series & Seitz, Reference Seriès and Seitz2013; Summerfield & de Lange, Reference Summerfield and de Lange2014). These predictions streamline cognitive processing by establishing mental frameworks that filter and prioritize information, enhancing our ability to navigate the physical and social environment. Extensive research has shown that expectations play a crucial role in various aspects of perception. For instance, expectations affect motion perception, leading to more accurate and faster responses to expected motion directions (Alink et al., Reference Alink, Schwiedrzik, Kohler, Singer and Muckli2010; Kveraga et al., Reference Kveraga, Boshyan and Bar2007). Similarly, expectations influence colour perception, making us perceive colours closer to what we predict (Olkkonen et al., Reference Olkkonen, Hansen and Gegenfurtner2008; Witzel & Gegenfurtner, Reference Witzel and Gegenfurtner2013). Other visual features, such as contrast and brightness, are shaped by expectations, demonstrating the extensive impact of predictive mechanisms on perception (see Summerfield & de Lange (Reference Summerfield and de Lange2014) for a review).

5.2 Structural and Contextual Expectations

Expectations can be categorized into structural and contextual types (Series & Seitz, Reference Seriès and Seitz2013). Structural expectations are formed through long-term interactions with the environment and result from implicit learning of the statistical regularities of the world around us. For example, consider the circles with light and dark shading in Figure 6. Circles with light at the top appear convex, while those with dark at the top look concave. This phenomenon occurs because the visual system assumes that light comes from above, an expectation based on everyday experiences with natural lighting (Bar, Reference Bar2004). Structural expectations are the ‘default’ expectations of the information processing system and are either hardwired or based on implicit learning of the statistics of the natural environment (Series & Seitz, Reference Seriès and Seitz2013).

Figure 6 The light from above assumption.

Moreover, we have a bias towards recognizing cardinal orientations – up, down, left, and right – more accurately than oblique angles (Girshick et al., Reference Girshick, Landy and Simoncelli2011). This orientation bias influences how we orient ourselves and navigate spaces, impacting everything from architectural design to the layout of digital interfaces. Another interesting structural expectation is the brain’s tendency to perceive objects as convex and backgrounds as homogeneously coloured (Goldreich & Peterson, Reference Goldreich and Peterson2012), a heuristic bias that simplifies the variegated array of visual stimuli we encounter, allowing us to quickly discern objects from their surroundings.

Studies show that these structural priors are not entirely immutable and can be reshaped through experience. For instance, Adams et al. (Reference Adams, Graf and Ernst2004) demonstrated that the ‘light-from-above’ prior could be altered through visual-haptic training. Participants exposed to specific lighting conditions adapted their internal representations of light direction, which generalized to other tasks, indicating that the prior is flexible and influenced by experiential learning. This research demonstrates the dynamic nature of perceptual systems, revealing how even deeply ingrained visual expectations can be modified through interaction with the environment.

Contextual expectations, on the other hand, are rapidly modifiable and can be influenced by instructions, sensory cues, or the context in which a stimulus is shown. Some of the earliest research on contextual expectations comes from experiments using bistable figures (Bugelski & Alampay, Reference Bugelski and Alampay1961). In these experiments, participants were shown bistable figures (e.g., the rat/man image). Before viewing them, some participants were primed with pictures or words related to one of the possible interpretations. The results indicated that expectations, driven by prior experiences of human faces or small mammals, determined what was perceived.

One way that expectations manifest is through context frames. These frames represent sets of expectations about the environment that are triggered rapidly by either global scene information or key objects within a scene (Bar, Reference Bar2004). They are populated with prototypical information about a given context, including the identities and typical spatial arrangements of objects that frequently co-appear. This mechanism allows the brain to predict what it will likely encounter next in the visual field and facilitates quicker and more efficient responses. For instance, seeing a steering wheel triggers expectations for the positions of other car-related elements, like the dashboard, radio, and mirrors, based on a stereotypical understanding of car interiors. These expectations are dynamic and continuously updated based on ongoing sensory input. The rapid activation of context frames is essential for managing the vast amount of visual information the brain processes, enabling it to focus on anomalies or unexpected features by comparing incoming data against these frameworks. This is highly adaptive as it conserves cognitive and neural resources and enhances perceptual efficiency, allowing the system to avoid repeatedly analysing every aspect of a familiar scene (Bar, 2004; Bar & Ullman, Reference Bar and Ullman1996).

A classic study by Palmer (Reference Palmer1975) demonstrated the effect of contextual expectations on perception, specifically how the perception of objects is influenced by the scenes in which they are embedded. In his experiments, participants were shown objects within appropriate contextual scenes (e.g., a toaster in a kitchen), inappropriate scenes (e.g., a printer in a kitchen), or no context. The results revealed that congruent objects (e.g., the toaster in the kitchen) were identified more quickly and accurately than incongruent objects (e.g., the printer in the kitchen).

In a series of experiments, Biederman et al. investigated the role of scene coherence on object detection (Biederman, Reference Biederman, Mezzanotte and Rabinowitz1982; Biederman et al., Reference Biederman, Rabinowitz, Glass and Stacy1974). In one experiment, participants viewed coherent and jumbled scenes to determine which object occupied a given cued position immediately after the scene was presented. Results indicated that participants were more accurate at identifying objects in coherent scenes than in jumbled scenes, even when they knew what objects to look for and where to look. This suggested that jumbling primarily affected perceptual recognition rather than memory or response selection (Biederman et al., Reference Biederman, Glass and Stacy1973). They then extended these findings by examining the time required to identify objects in coherent versus jumbled scenes (Biederman et al., Reference Biederman, Rabinowitz, Glass and Stacy1974). Participants were given a picture of a target object before viewing the scene and were then asked to judge whether the target object was present. The results showed slower detection times for objects in jumbled scenes, especially when the target object was not present but likely to occur.

In another study, they explored the effects of disrupting five semantic and physical relations that define a coherent scene on object perception: the probability for an object to appear in the scene, the position of the object within the scene, the relative size of the object, whether it was supported or not, and whether it was occluded (Biederman, Reference Biederman, Kubovy and Pomerantz1981; Biederman et al., Reference Biederman, Mezzanotte and Rabinowitz1982). By manipulating these relations, the researchers could measure their effect on object identification. For example, one would expect to see a fire hydrant in the street but not in a kitchen, which would be a violation of probability – the likelihood of that object appearing in that context. One would also expect to see a chair on the floor and not on the ceiling (a violation of the position relation), a cup smaller than a kitchen table and not the opposite (a size violation), a sofa resting on the ground and not floating in the air (a violation of support), and a wall becoming occluded as a cat moves past it, not for the wall to remain visible through the cat (a violation of occlusion). In one experiment (Biederman et al., Reference Biederman, Mezzanotte and Rabinowitz1982), participants viewed scenes (150 ms) in which a cued object was either congruent with the scene or not. They found that, except for the occlusion relation, participants were less accurate and slower in detecting cued objects when these semantic and physical relations were violated. Interestingly, detecting a non-violating stimulus was not affected by the presence of another object undergoing a violation within the scene. This shows the powerful influence of scene schema in the perception and identification of objects within a scene.

Further support comes from Davenport and Potter (Reference Davenport and Potter2004), who found that an object was identified more quickly when presented with a related background (e.g., a priest in a church) compared to an unrelated background (e.g., a priest in a football stadium). In this study, objects in isolation were identified most quickly. However, when there was a background to the object, the meaning of the context and the gist of the scene significantly impacted the reaction time for participants to name the target. The results led the authors to conclude that objects are not processed independently of their surroundings but interact with the processing of the scene (see also Oliva & Torallba (Reference Oliva and Torralba2007)), such that visual processing of the object is influenced by the scene gist.

Munneke et al. (Reference Munneke, Brentari and Peelen2013) examined whether the scene consistency effect on object recognition is influenced by focused attention. They used a location cueing method where participants were informed about the target object’s location on some trials, allowing them to direct their attention accordingly. The study found that scene consistency effects were independent of spatial attention, occurring whether participants’ focus was on the target or the background. This suggests that consistent scene contexts aid object recognition regardless of attention, likely driven by global scene properties or ‘scene gist’, which are processed with minimal attention.

Further insights come from neuroimaging studies examining the modulation of object representations by scene context. In one experiment, degraded images of objects were shown either in isolation or within congruent scenes, such as a helicopter in the sky. fMRI results indicated that the presence of a scene context enhanced the neural representations of these objects, making the activity patterns in the object–selective cortex resemble those evoked by intact objects more closely. This neural sharpening effect was correlated with concurrent activity in scene-selective areas, suggesting that scene context provides predictive signals that enhance object processing (Brandman & Peelen, Reference Brandman and Peelen2017).

These findings collectively emphasize the influence of scene context on object perception. Scene context, or the gist of the scene, provides a predictive framework shaped by prior experiences and expectations to more accurately and efficiently identify and interact with the objects within it. According to the Spatial Envelope Model (Oliva & Torralba, Reference Oliva and Torralba2006), a scene has its own ‘shape’: a gestalt formed by global properties, such as the degree of naturalness, openness, expansion, roughness, and ruggedness. A forest, for example, has a greater degree of naturalness than a city square. One would expect irregular shapes and the organic texture of the leaves and trees in the former. In contrast, one would expect more horizontal and vertical edges and more geometric manufactured shapes in the latter. Scene gist is thus seen as a statistical summary of the scene, one which can be rapidly determined by global properties and which influences perception of the objects within that scene (Hollingworth & Henderson, Reference Hollingworth and Henderson1999; Potter et al., Reference Potter, Wyble, Hagmann and McCourt2014).

The psychophysical and neuroimaging findings on scene and object perception can be explained through a predictive processing framework. Peelen et al. (Reference Peelen, Berlot and de Lange2024) have developed such a model that emphasizes the bidirectional influences between scene and object perception. In their model, visual processing pathways for objects and scenes operate in parallel, with object processing focusing on detailed, high-resolution foveal input and scene processing on larger-scale, low-resolution peripheral input. These pathways interact hierarchically, where higher levels generate predictions to guide lower-level computations. In this predictive processing model, scene and object information is integrated through precision-weighted Bayesian inference. This means that the brain combines multiple sources of information, each weighted by its reliability, to form a coherent percept. For example, scene context, which is typically more reliable, can strongly guide object recognition. This hierarchical structure allows for efficient disambiguation of ambiguous sensory input, as higher-level scene representations provide predictive signals that enhance the neural representations of objects in object-selective areas and vice versa.

5.3 Neural Mechanisms of Expectation

Recent research is revealing the neural mechanisms underlying the influence of expectations on sensory processing. For example, Kok et al. (2011) explored how predictions influence processing in the primary visual cortex (V1). Using fMRI and multivariate pattern analysis (MVPA), they demonstrated that perceptual expectation reduced neural response amplitude in V1 while improving the specificity of stimulus representation. This suggests that the brain uses expectations to sharpen sensory representations, facilitating efficient and accurate perception.

Kok et al. (Reference Kok, Brouwer, van Gerven and de Lange2013) further investigated how top-down expectations bias sensory representations in the visual cortex. They found that expectations significantly influenced neural activity in early visual cortex areas such as V1, V2, and V3. The direction of motion reconstructed from the BOLD signal was biased in the direction predicted by auditory cues, demonstrating that expectations alter perception and underlying neural representations.

In another study, Kok et al. (Reference Kok, Failing and de Lange2014) examined how prior expectations shape sensory representations in the primary visual cortex. Using fMRI, they showed that expectation of a visual stimulus evokes a feature-specific pattern of activity in V1 similar to that evoked by the actual stimulus. This pre-activation of stimulus-specific patterns suggests that the brain uses prior knowledge to prepare sensory areas for expected inputs, enhancing perceptual accuracy and efficiency.

Egner et al. (Reference Egner, Monti and Summerfield2010) explored how expectation and surprise influence neural responses in the fusiform face area (FFA). Their fMRI study showed that high expectations for faces led to reduced neural responses when faces were presented, whereas unexpected faces elicited stronger responses. This supports the predictive coding model, where neural responses are shaped by both the prediction and the prediction error.

These findings are supported by Bayesian theories which posit that perception is biased towards expectations to optimize veridicality, increasing the gain on expected relative to unexpected inputs. In contrast, so-called Cancellation theories argue that perception is geared more towards unexpected signals to optimize informativeness, by suppressing expected sensory activity (Press et al., Reference Press, Kok and Yon2020). The two-process model proposed by Press et al. (Reference Press, Kok and Yon2020) resolves this paradox by suggesting that perception initially biases towards expected inputs to rapidly generate veridical experiences. However, when inputs significantly deviate from predictions, generating high levels of surprise, a secondary process boosts these unexpected inputs. As predicted by the model, some studies show that expected events are perceived more intensely 50 ms post-stimulus, with this bias inverting by 200 ms to unexpected events (Yon & Press, Reference Yon and Press2017). Further supporting this, EEG research on infants has observed initial enhanced processing of expected events, which later transitions to a preference for unexpected events (Kouider et al., Reference Kouider, Long and Le Stanc2015).

The neural mechanisms of expectation, as revealed by these research findings, add support to the brain as a dynamic prediction machine, continually refining its rich hierarchical representation of the world to navigate the uncertainties of the environment, where minimizing surprise or free energy is crucial (Friston, Reference Friston2010). Expectations inform what we perceive in the present moment. What you are perceiving right now is constructed from higher-level expectations (say, about an independent spatio-temporal world, object permanence), perhaps hardwired or learned developmentally and resistant to being updated; mid-level expectations about the dynamics of the current environment, about the street you are driving along, how these cars are behaving, how these people are behaving, and what they will do next; and lower-level expectations about the ever-changing tapestry of sensory information, about edges, motion, shape, depth, texture, and colour. The result is a perception of the world, sculpted by the interplay between predictions at each level and precision-weighted prediction errors.

In the last two sections, we have examined the roles of attention and expectation in perception.

We have reviewed evidence demonstrating the necessity of attention for conscious perception and the powerful influence of expectation on what we perceive. In the next section, we will explore research on the relationship between attention and expectation.

6.1 Introduction

As we have seen, attention and expectation play significant roles in shaping perceptual experiences, influencing how we process and interpret sensory information. This section explores studies that examine how attention and expectation interact.

6.2 Expectation Violations and Attention Capture

Some experiments conducted in our lab (Mack et al., Reference Mack, Clarke, Erol and Bert2017) were guided by critical questions about the role of expectation violations in capturing attention. Earlier research had revealed that incongruities within a scene can unconsciously capture attention, leading to awareness of the scene. Mudrik et al. (Reference Mudrik, Breska, Lamy and Deouell2011) conducted studies demonstrating that scenes depicting incongruent actions, such as a woman baking a chessboard instead of cookies, can slow responses to subsequent scenes, whether congruent or incongruent.

In one of their studies, they presented masked scenes of a person performing an action with either a congruent or an incongruent object (e.g., a man pouring coffee into a mug versus a roll of toilet paper). These were followed by briefly presented target scenes containing either a congruent or incongruent object, and participants were tasked with judging the congruency of these targets as quickly as possible. The study found that reaction times were longer when targets were preceded by scenes with incongruent objects, even though the scenes were masked and not consciously perceived. This suggests that the brain processes relationships between objects and their contexts outside of consciousness, requiring additional cognitive resources to resolve incongruities. The invisibility of the masked scenes was confirmed through subjective and objective measures, eliminating the possibility of partial awareness. These results indicate that incongruent elements within a scene can be processed unconsciously, affecting the processing speed of subsequent stimuli (Mudrik & Koch, Reference Mudrik and Koch2013). Furthermore, in studies involving binocular rivalry, Mudrik et al. (Reference Mudrik, Breska, Lamy and Deouell2011) found that incongruent scenes escape perceptual suppression faster than congruent ones and dominate visual consciousness for longer durations. This suggests that the brain allocates more attentional resources to resolve these incongruities, which are difficult to integrate without conscious processing (Mudrik et al., Reference Mudrik, Breska, Lamy and Deouell2011).

6.3 Experimenting with Scene Incongruity

Inspired by these results, Mack et al. (Reference Mack, Clarke, Erol and Bert2017) conducted experiments using the same stimuli in Mudrik et al.’s (Reference Mudrik, Breska, Lamy and Deouell2011) experiments to investigate whether scene incongruity captures attention and leads to conscious perception under four conditions: inattention, full attention, change detection, and IM.

In the first experiment, participants were exposed to scenes with congruent or incongruent objects (3.8 by 4.7 degrees of visual angle) under inattention, divided attention, and full attention conditions, with scenes presented for 100 ms followed by a pattern mask. In the inattention condition, 60 per cent of participants reported only the cross, and those who noticed the scenes often normalized incongruent objects, such as mistaking a girl licking a light bulb for eating ice cream. With divided attention, 70 per cent noticed scenes but normalized incongruities, while in the full attention condition, 90 per cent noticed scenes but did not identify incongruities, normalizing them instead. Increasing the scene size to 5.7 by 7.05 degrees did not significantly improve detection of incongruities, as 40 per cent in the inattention condition, 90 per cent in the divided attention condition, and 80 per cent in the full attention condition still normalized incongruent objects. Extending the presentation time to 200 ms also did not improve detection; 50 per cent in the inattention, 70 per cent in the divided attention, and 90 per cent in the full attention condition noticed scenes but normalized incongruities. Subsequent experiments involving classifying scenes as ‘weird’ or ‘not weird’ and using the flicker paradigm showed that participants often failed to detect incongruities, instead normalizing them. Finally, testing interference of incongruent elements in visual arrays showed no significant perceptual sensitivity differences, with only one participant noticing anything unusual.

These findings indicate that scene incongruity does not reliably capture attention for conscious perception, as participants frequently normalized incongruent objects based on scene gist, even with full attention. Increasing scene size and presentation time did not significantly improve detection of incongruities. In contrast, Clarke and Porubanova (Reference Clarke and Porubanova2020) found that scenes with semantic violations are perceived as lasting longer, indicating that unexpected features in scenes require more cognitive processing time, leading to subjective time dilation. Their study, which involved larger stimuli presented for longer durations, revealed that processing incongruities increases perceived time. Tachmatzidou and Vatakis (Reference Tachmatzidou and Vatakis2023) further explored the impact of semantic and syntactic violations on time perception, showing that semantic violations led to time compression, while syntactic violations led to time dilation. They also found that increasing the contrast of target objects amplified these effects, highlighting the role of attention in modulating duration perception. The differing results between Clarke and Porubanova (Reference Clarke and Porubanova2020) and Tachmatzidou and Vatakis (Reference Tachmatzidou and Vatakis2023) regarding semantic violations may stem from methodological differences. Moreover, the discrepancy between Mack et al. (Reference Mack, Clarke, Erol and Bert2017) and Clarke and Porubanova (Reference Clarke and Porubanova2020) could be attributed to the size and duration of the stimuli used, suggesting that these factors play a crucial role in the perception of incongruent scenes. Further investigation is needed to understand the underlying mechanisms of how different types of violations and various stimulus characteristics affect attention and perception, and to clarify the roles of scene size, duration, and contrast in these processes. This could involve systematically varying these parameters to discern their individual and combined effects on the conscious perception of scene incongruities.

6.4 Evidence from Cognitive Neuroscience

Jiang et al. (Reference Jiang, Summerfield and Egner2013) investigated the interaction between attention and expectation in visual perception using fMRI. Participants engaged in tasks with manipulated expectations and attention, where they viewed faces and scenes paired with predictive auditory cues. They found that attention enhances the precision of prediction errors, making expected and unexpected stimuli more distinct at the neural level. The results indicated that attention and expectation enhance category selectivity through different neural mechanisms, with attention affecting higher cortical levels and expectation modulating lower cortical levels.

Gordon et al. (Reference Gordon, Tsuchiya, Koenig-Robert and Hohwy2019) used EEG and hierarchical frequency tagging (HFT) to explore how attention and expectation modulate the integration of top-down and bottom-up signals in visual perception. They revealed that attention and expectation influence signal integration through different neural pathways, with expectation affecting descending signals and attention modulating ascending signals. This study provides physiological evidence that attention and expectation optimize perceptual processing via distinct mechanisms.

Expectation, in this context, primarily influences descending signals, meaning it shapes the predictive frameworks the brain uses to process incoming sensory information. This top-down modulation is consistent with the idea that the brain constantly generates and updates models to predict sensory input, thereby facilitating perceptual processing when the actual input aligns with these predictions (Clark, Reference Clark2013; Hohwy, Reference Hohwy2013).

On the other hand, attention modulates ascending signals, meaning it enhances the processing of incoming sensory information by prioritizing relevant stimuli. This modulation aligns with extensive neuroscientific evidence showing that attention can amplify neural responses to attended stimuli, thereby enhancing their salience in the perceptual processing stream (Kastner & Ungerleider, Reference Kastner and Ungerleider2000; Martinez et al., Reference Martínez, Teder-Salejarvi and Hillyard2007; Mehrpour et al., Reference Mehrpour, Martinez-Trujillo and Treue2020).

Duyar et al. (Reference Duyar, Ren and Carrasco2024) explored the interaction between temporal attention and expectation in visual perception. Their study demonstrated that voluntary temporal attention and expectation jointly enhance perceptual accuracy and neural responses. Using tasks requiring participants to focus on specific time intervals and predict visual stimuli, they found that attention and expectation synergistically improve the processing of events, revealing their combined importance in optimizing perceptual performance.

The evidence suggests that attention generally enhances sensory inputs through the facilitation of ascending signals, while expectation shapes perception by refining the brain’s predictive models, thus affecting descending signals (Clark, Reference Clark2013; Friston, Reference Friston2005). However, it is important to note that attention not only facilitates but also inhibits processing. For example, Reynolds and Heeger (Reference Reynolds and Heeger2009) found that attention can suppress the processing of irrelevant stimuli, thereby reducing their interference with the attended information. Therefore, it is clear that both mechanisms are integral to attention and expectation. Research indicates that attention involves both facilitatory and inhibitory processes depending on the context and task demands (Carrasco, 2011; Reynolds & Heeger, Reference Reynolds and Heeger2009). This perspective is consistent with the broader body of behavioural and neuroscientific data on the top-down effects of attention, which enhance perceptual sensitivity and cognitive processing efficiency (e.g., Jiang et al., Reference Jiang, Summerfield and Egner2013; Carrasco et al., Reference Carrasco, Ling and Read2004).

7.1 Expectation Causes Illusory Perception without Attention

Expectations can lead us to see what is not there, even without attention. This section examines the evidence supporting this phenomenon and discusses experimental research showing how expectations may cause illusory perceptions in the absence of stimuli.

Mack et al. (Reference Mack, Erol, Clarke and Bert2016) explored the role of attention in IM and its relationship with expectations. This study aimed to determine whether IM can exist without attention. In this experiment, participants were presented with a matrix of letters at fixation and the circle task described earlier. The probability of reporting whether the four circles were the same or whether one was different was 90 per cent, which led the participants to attend more to the circle task than reporting the letters in the matrix. This demanding task allowed the researchers to explore what is perceived without attention. On the critical 101st trial, with their attention maximally diverted from the matrix, participants were instructed to report letters from the matrix, which were, in fact, not presented. More than half of the participants reported seeing a matrix of letters not present on the screen. When asked to enter the letters they ‘saw’, participants wrote down letters, suggesting an illusory perception.

The researchers explored this in another study (Erol et al., Reference Erol, Mack, Clarke and Bert2016). Participants reported the longer arm of a cross in the visual periphery while a colour-filled circle, alternating between blue and yellow from trial to trial, appeared at fixation on every trial except for the critical trials. On the critical trial, only the cross appeared. Participants were tested under inattention, divided attention, and full attention conditions. In the inattention condition, after twenty-five trials with the cross and coloured circle, only three of the fifteen participants reported the absence of the coloured circle, while the remaining twelve reported seeing either a blue or yellow circle despite its absence. In the divided attention condition, participants reported both the longer arm of the cross and anything else they noticed. Seven of the fifteen participants in the critical trial were unaware of the circle’s absence and incorrectly reported seeing it. In the full attention condition, participants were instructed to ignore the cross and report anything else on the screen. Here, fourteen of the fifteen participants correctly identified the absence of the circle on the critical trial. This significant difference in awareness between the inattention and full attention conditions demonstrates a powerful effect of expectation in visual perception: the illusion of seeing a stimulus even when it is not present.

In another study, Erol et al. (Reference Erol, Mack and Clarke2018) explored this phenomenon with a highly meaningful stimulus – a face. In non-critical trials, participants reported whether four colour-bisected circles surrounding a face at fixation were the same or different. On the final critical trial, the face was absent. They found that 73.3 per cent of participants in the inattention condition reported seeing a face when it was absent, 46.7 per cent in the divided attention condition, and 0 per cent in the full attention condition. Conversely, when the face was present, 66.7 per cent of participants in the inattention condition, 93.3 per cent in the divided attention condition, and 100 per cent in the full attention condition reported seeing it. These findings illustrate the significant influence of unintentionally formed expectations on perception, leading to illusory perceptions of faces even when they are not present.

Aru and Bachmann (Reference Aru and Bachmann2017) conducted a study to replicate and extend the findings of Mack et al. (Reference Mack, Erol, Clarke and Bert2016). They used the Perceptual Awareness Scale (PAS), which measures the subjective visibility of letters. This scale allowed participants to rate their experience of the letter array from 1 (‘no experience of the stimulus’) to 4 (‘clear impression of the stimulus’). In the dual-task condition, where attention was primarily diverted to the circles (cued 90 per cent of the time), on the 101st trial, no letters were presented, yet participants rated their visibility. The results revealed that six out of seventeen participants did not notice the absence of letters and rated the visibility of the non-existent letters similarly to trials where letters were present. These participants gave an average rating of 2.7 for the non-existent letters, indicating an ‘almost clear impression of the stimulus’.

In a later study, Aru et al. (Reference Aru, Tulver and Bachmann2018) developed dual-task setups to investigate how expectations might lead to illusory perceptions without attention. Across three experiments, participants engaged in a primary task while occasionally being queried about an auxiliary task designed to induce expectations. In the first experiment, participants viewed faces with surrounding squares and rated the visibility of these squares. Despite the squares being absent in critical trials, over 90 per cent of participants reported seeing them at least once, with many giving high visibility ratings. Only one participant never rated the visibility higher than 1 (‘no experience of the stimulus’) across the six critical trials. On average, participants reported some experience of the missing stimulus in 50 per cent of the critical trials, with six of the fourteen participants perceiving illusory squares on more than three occasions, and nine participants rating the visibility as ‘almost clear’ at least once. Interestingly, the study found a negative correlation between Autism Spectrum Quotient (AQ) scores and the frequency of these illusory perceptions, suggesting that individuals with higher autistic traits were less susceptible to such illusions.

In the second experiment, participants were asked to discriminate between the orientation of the faces or report the visibility of a small Landolt square, a figure consisting of a square with a gap on one side, and which was absent on critical trials. The findings were consistent with the first experiment, as 67 per cent of critical trials involved reports of experiencing the absent stimulus. Twelve out of fifteen used the rating ‘almost clear expression’ for the absent stimulus at least once, and six participants reported a ‘clear impression’ of the missing stimulus. This demonstrated a strong effect of expectations on perception, even in a setup designed to divert attention from the auxiliary task.

The third experiment further supported these findings. Participants had to identify odd-coloured circles or rate the visibility of non-existent letters. Out of the seventeen participants, three did not experience any illusory perception, but six participants had illusory perceptions in more than half of the trials. Seven participants rated the visibility as ‘almost clear’ at least once, and four reported a ‘clear impression’ of the missing stimulus on at least one trial.

These experiments consistently showed that participants often reported perceiving stimuli that were not present, with varying degrees of illusory perception influenced by task difficulty and attentional demands. Notably, even under conditions designed to divert attentional focus from the auxiliary task, expectations still led to high rates of illusory perceptions. The results suggest that even when no letters, colour patches, or faces were present, participants’ brains ‘filled in the gap’ with expected content, leading to the perception of a stimulus that was not there. These findings reveal the power of experimentally induced expectations in creating vivid illusory perceptions in the absence of attention: expectation awareness. Both IB and expectation awareness can be explained by the predictive processing framework. This is the topic of the next section.

7.2 Expectation Awareness and Inattentional Blindness: A Simple Predictive Processing Model

Expectation awareness and IB reveal the interwoven dynamics of attention and expectation in shaping perceptual experiences. This section will examine how predictive coding can explain the phenomena of IB and expectation awareness.

7.2.1 Expectation Awareness: Predictive Coding Explanation

Expectation awareness arises within the predictive coding framework when high precision is assigned to predictions and low precision is assigned to sensory inputs. When participants expect a stimulus, the brain generates strong prior predictions (high precision on priors). If attention is focused elsewhere, the precision of sensory input from the expected location is low. Consequently, any prediction errors resulting from the absence of the stimulus are given little weight, and the strong prior prediction dominates perception, leading to an illusory experience (Hohwy, Reference Hohwy2012). This can be mathematically illustrated with the following equations:

Prediction Error Calculation

Prediction Error Calculation:

$ϵ=y–\text{μ}$

Precision-Weighted Prediction Error:

$ϵ̃=\Pi ϵ$

Updating Predictions:

$\Delta \mu =\eta \epsilon ̃$

Here, y represents sensory input, μ represents the prediction, Π represents precision, and η is the learning rate. $ϵ̃$ is precision-weighted prediction error.

In the case of expectation awareness, Π is low for sensory input and high for predictions, resulting in a perceptual experience dominated by the prior prediction (see Figure 7).

Figure 7 Expectation awareness. This illustrates how inattention and prior expectations can lead to perceiving stimuli that are not present.

7.2.2 Inattentional Blindness: Predictive Coding Explanation

Predictive coding explains IB by focusing on the low precision assigned to unattended sensory inputs. When attention is directed towards a task, the precision of sensory input related to that task is high, ensuring accurate perception of task-relevant stimuli. However, the precision of unattended sensory inputs is low, reducing the impact of prediction errors and leading to failures in noticing unexpected stimuli (Dehaene et al., Reference Dehaene, Changeux, Naccache, Sackur and Sergent2006). This can be expressed mathematically as follows:

When attention is high on the task:

${\widetilde{ϵ}}_{high}^{}={\text{Π}}_{high}\cdot (y-\mu )$

When attention is low on unexpected stimuli:

${\widetilde{ϵ}}_{low}^{{}_{}}={\text{Π}}_{low}\cdot (y-\mu )$

Here, Π is high for task-relevant inputs, resulting in a strong updating of predictions based on these inputs. Conversely, Π is low for unexpected stimuli, leading to minimal updating of predictions and IB (see Figure 8).

Figure 8 Inattentional blindness. This illustrates how focused attention on a task can lead to missing unexpected objects.