Neural bases of eye and gaze processing: The core of social cognition

2.1. Eyes are central to various aspects of face processing

Studies monitoring ocular movements during face perception have shown that people spend more time on internal features (eyes, nose, mouth) than external ones (hair, face contour, forehead, ears) (Althoff and Cohen, 1999; Walker-Smith et al., 1977; Yarbus, 1967). Such findings support the idea that internal features play a central role in face perception and recognition, especially for familiar faces (Ellis et al., 1979). Numerous studies have shown that the eye region is the most attended of all facial features and the source of information the most utilized regardless of the task, whether it focuses on gaze, head orientation, identity, gender, facial expression or age (e.g. Henderson et al., 2005; Itier et al., 2007c; Janik et al., 1978; Laughery et al., 1971; Luria and Strauss, 1978; Schyns et al., 2002). This attraction to the eyes is even more pronounced for familiar faces (Althoff and Cohen, 1999).

Like faces, eyes vary greatly from one individual to another and the eye region may in fact be the facial zone that varies mostly between people. This inter-subject variability has been investigated by many morphological and biometrical studies, especially in the field of Anthropology (Farkas, 1994; Hall et al., 1989). Eye color and shape, and inter-ocular, inter-canthal and inter-pupillar distances are specific to each individual. Other elements of the eye region such as eyebrows, eyelids, eyelashes and their respective distances constitute numerous configural cues necessary for the recognition of face identity and for this reason are used in robot-portraits, the computerized drawing representations of individual faces obtained from descriptions of their various features. For instance, 13 measures of the eye region are utilized in forensic identification in which the face of missing people is reconstructed based on old photographs or even skulls (Farkas, 1994; Farkas et al., 1994). When the eye region is masked, face recognition performances drop while masking the nose or the mouth has little or no effect (McKelvie, 1976). Familiar face recognition performances drop even more so when eyebrows, rather than the eyes, are removed from the picture (Sadr et al., 2003). Similarly, face detection is disproportionately impaired when the eye region is occluded compared to when the nose, mouth or forehead are occluded (Lewis and Edmonds, 2003). Image classification techniques have also shown that the eye region is the diagnostic feature used to discriminate gender (Schyns et al., 2002; Vinette et al., 2004) and to recognize identity (Caldara et al., 2005), i.e. it is the principal element subjects use to decide whether a face is male or female or who it is. When noise is added to the picture, identity discrimination between two faces is performed using the eye region including eyebrows (Sekuler et al., 2004). The eye region is thus a key element of face recognition.

In addition to its important role in processing identity, the eye region carries information necessary for emotion recognition (Calder et al., 2000; Ekman and Friesen, 1978; Fox and Damjanovic, 2006; Smith et al., 2005) and is thus central to non-verbal communication. Fear and surprise are characterized by wide open eyes and by a larger white sclera size (Whalen et al., 2004) and masking the eye region results in a drop of fear recognition performances (Adolphs et al., 2005). The inferior eyelid is contracted when the person is expressing fear but relaxed when expressing surprise (Ekman and Friesen, 1978). In faces expressing disgust however, the eyes are squinted. Joy is mostly characterized by the inferior part of the face, especially by the mouth (smile) which is the diagnostic element enabling the recognition of this emotion (Schyns et al., 2002). However, a fake smile will be betrayed by the absence of expressive cues in the eye region, such as the wrinkles around the eye corners or the squinting of the eye opening by the ocular muscles (the so-called “Duchenne smile”, Duchenne, 1990; Ekman, 1992). A recent study showed that, although not affecting accuracy rates, ocular cues influence the speed of joy recognition but this depends on the task context (Leppänen and Hietanen, 2007). Anger is implied by the frowning of the eyebrows and other eye cues (Calder et al., 2000; Smith et al., 2005), and sadness by a down-looking gaze (Ekman and Friesen, 1978). Thus, all six basic emotions described by Ekman (joy, fear, anger, sadness, surprise and disgust, Ekman and Friesen, 1971) involve a specific change in one element of the eye region. Even though, as measured by one image classification technique, the eye region is the diagnostic element utilized to recognize fear (Schyns et al., 2007), the scanning of the face always starts with the eyes regardless of its emotion (Schyns et al., 2007; Vinette et al., 2004), again supporting a role of the eye region in processing all facial expressions. Finally, isolated eye regions are often sufficient to recognize the six basic emotions but also more complex feelings such as jealousy, envy or guilt (Baron-Cohen et al., 1997b, 2001a). The eyes, a “window to the soul”, inform us on the emotion and the state of mind of others, a topic we return to in Section 3.2.

The eye region thus attracts attention and represents a special area of the face from which extensive amount of information can be extracted, such as identity and emotion cues. We now turn to another important piece of information derived from the eye region, gaze direction. Before dealing with the role of gaze direction in attention orienting (Section 3.1), we first describe below the influence of gaze on various processes such as the perception of gender, identity and facial emotions.

2.2. Gaze direction in face perception, gender discrimination, identity recognition and facial expression discrimination

The systematic attraction of attention towards the eyes of a face reviewed above, starts early in development (Maurer, 1985) and seems linked to the perception of gaze. Newborns prefer to look at a face with open rather than closed eyes (Batki et al., 2000) and look longer at faces whose gaze is directed at them compared to averted-gaze faces (Farroni et al., 2002). Three-month-old infants also smile less when an interacting person gazes away after having made eye contact (Hains and Muir, 1996). If gaze contact is often perceived as a threat in most species (Emery, 2000), its meaning has evolved in humans and the early attraction of newborns to direct gaze is linked to social communication (Farroni et al., 2002). In everyday life, a direct gaze signals a potential social interaction (positive or negative) while an averted gaze implies that the person is attending to something or someone else than us. For instance, a recent study showed that direct gaze is related to approach behavior while averted gaze is related to avoidance, but this is found only when real persons are part of the study design rather than pictures of faces (Hietanen et al., 2008b). In humans, the power of gaze is due to its social impact (Kleinke, 1986).

Because of the spontaneous attraction of attention to the eye region, it seems reasonable to believe that information in this area, and in particular gaze direction, could influence other processes related to faces. Several behavioral studies have shown that gaze direction can influence person perception and recognition but findings are inconsistent. For instance, gender categorization was faster for direct than averted-gaze faces in one study (Macrae et al., 2002) while the opposite was found in another study but only when the face was in 3/4-view (Vuilleumier et al., 2005). This gaze effect on gender categorization was also more pronounced when the face was of opposite gender to that of the subject (Vuilleumier et al., 2005). This difference in the two studies could be due to the stimuli used. Vuilleumier et al. (2005) used a fully counterbalanced design between head orientation (front- and 3/4-view) and gaze direction (direct or averted) while Macrae et al. (2002) used 3/4- and front-view faces with direct gaze but only 3/4-view-faces with averted gaze.

In contrast, using the same type of stimuli as Vuilleumier et al. (2005) but an explicit gaze direction judgment task, one study reported faster response times only for direct-gaze-front-view faces (Pageler et al., 2003). More recently, using an explicit gaze direction judgment and a head orientation judgment along with the same kind of stimuli, Itier and colleagues reported a true interaction between gaze direction and head orientation, with faster response times for congruent conditions and longer response times for incongruent conditions when face orientation and gaze direction did not match (Itier et al., 2007a,c). These behavioral results were found on two different subject groups and replicate previous findings (Langton, 2000). The reasons for these inconsistent results in the literature concerning gaze direction and head orientation interactions are still unclear but could be due, in addition to task effects, to the use of slightly different paradigms adapted to the different methodologies used (e.g. ERPs and eye tracking in Itier et al., 2007a,c; fMRI in Pageler et al., 2003; strictly behavioral tests in Vuilleumier et al., 2005 and in Langton, 2000). In another recent gaze direction discrimination task with similar stimuli, subjects were faster to detect eyes moving towards the viewer (direct gaze motion) rather than moving away from the viewer (averted gaze motion) and this was found regardless of head orientation, although the effect was more pronounced for front-view faces (Conty et al., 2007). In contrast to previous studies which used static faces, this paradigm involved gaze motion and this may explain the difference in results.

More consistent findings have been reported concerning the influence of gaze direction on identity encoding and recognition. It has been shown that direct-gaze faces are better encoded (Mason et al., 2004) and better recognized than averted-gaze faces, especially for 3/4-view faces of opposite gender to the subject (Vuilleumier et al., 2005). This effect of direct gaze on face recognition has been reported in newborns (Hood et al., 2003), 4-month-old infants (Farroni et al., 2007) and children from 6 to 11 years of age (Smith et al., 2006) for front-view faces and will have to be tested with 3/4-view faces. In one study, subjects were also faster to categorize letter strings as words when they were primed by a direct-gaze compared to an averted-gaze face regardless of head orientation (Macrae et al., 2002). The authors concluded that direct gaze facilitates the access to semantic information concerning a person (Macrae et al., 2002), although that study used unfamiliar faces and common words that were not specifically descriptive of people. Conversely, familiarity can impact on gaze processing as suggested by one study in which a recently encountered face (that thus became recently familiar) biased the perception of downward gaze direction which was perceived as more directed towards the subject compared to when the face was unfamiliar (Teske, 1988). Overall, it seems that gaze direction influences face categorization and recognition processes (and gaze perception may also be influenced by factors such as familiarity) but no clear effect of direct or averted gaze can yet be drawn from all these studies given the variability in task and stimuli used.

Gaze direction also seems to modulate the perception and the understanding of someone’s emotion. A special role for direct gaze in communicating increased emotional intensity regardless of the emotion was suggested in earlier work (Kleinke, 1986) while more recent work argues in favor of a differential role of direct and averted gaze depending on the emotion. For instance, one study reported that angry and joyful faces with direct gaze were categorized more quickly and more accurately than angry and joyful faces with averted gaze while the opposite was found for fearful and sad faces which were categorized faster and better when gaze was averted rather than direct (Adams and Kleck, 2003). In another study by the same authors, anger and joy were also perceived as more intense in direct-gaze compared to averted-gaze faces while fear and sadness were perceived as more intense for averted-gaze faces (Adams and Kleck, 2005). The authors suggested that direct gaze enhances the perception of approach-oriented emotions (anger and joy) while averted gaze enhances the perception of avoidance-oriented emotions (fear and sadness). Alternatively, these faster responses and increased perceived emotional intensity may be explained in terms of increased threat-related levels: an angry face looking straight at you may imply a coming danger, as you may be the object of the anger while a fearful face looking to the side may signal a coming danger on that side which you need to detect fast. However, this threat-related account would not explain the results found for joy and sadness. An effect of direct gaze on the perception of anger has also been reported in 4-month-old infants while the perception of other facial emotions does not seem to be modulated by gaze direction that early in development (Striano et al., 2006).

However, other studies failed to reproduce the original results of Adams and Kleck (Bindemann et al., 2008; Graham and LaBar, 2007). Bindemann et al. (2008) showed that emotion categorization varied as a function of the number of facial expressions included in a given paradigm and suggested that the results of Adams and Kleck likely reflected strategic task effects rather than real effects of gaze on facial expression categorization (Bindemann et al., 2008). Graham and LaBar (2007) also showed that gaze direction modulates expression processing only when facial expressions are difficult to discriminate (i.e. more ambiguous). These inconsistencies remain so far unclear but future studies will have to compare all facial expressions in each paradigm, including surprise which is rarely involved in gaze and emotion paradigms. We also want to point out that these studies used only front-view faces and that in addition to task context and baseline “discriminability” of facial emotions, face orientation could also influence facial expression categorization just like it influences other categorization tasks as reviewed above and this will also have to be investigated in the future.

Gaze also influences the perception of face attractiveness: faces with direct gaze are rated as more attractive than faces with averted gaze (Strick et al., 2008), and some have suggested a reward effect of direct gaze when the face is attractive but not when it is unattractive (Kampe et al., 2001). Furthermore, gaze direction influences the affective appraisals of objects in the environment as objects that are looked at by other people are liked more than objects that do not receive others’ attention (Bayliss et al., 2006). The affective appraisal of objects is further influenced by the combination of others’ gaze direction and their facial expression: objects are judged more pleasant when the person who is looking at them looks joyful rather than disgusted (Bayliss et al., 2007). However, these studies by Bayliss and colleagues used cuing paradigms (see Section 3) and these interaction effects between gaze, facial emotions and object appraisal should be verified in non-cuing studies. One study also reported that objects were evaluated more positively when they were associated with direct-gaze attractive faces but not with averted-gaze attractive faces (Strick et al., 2008). Finally, the association of others’ gaze direction with their facial expressions constitutes a social cue that seems to be utilized very early in development. As early as 1 year of age, infants use this information to judge whether they can approach and manipulate an object or whether it represents a danger and should be avoided (Mumme and Fernald, 2003).

The inconsistencies reported in the literature preclude any firm conclusion about the specific impact of direct and averted gaze on various cognitive processes related to faces. Factors like head orientation, gender of the subject and of the stimuli and task context seem to interact in complex ways with gaze direction. Finally, the use of photographs rather than real persons is another important factor that modulates the impact of gaze on face processing (Hietanen et al., 2008b) and the understanding of these complex interactions represents a challenge for future studies.

3.1. Orienting of attention by gaze

Another communicative function of eyes is to direct attention on specific places and objects of the environment through gaze. If someone is looking directly at us then we are the object of their attention. Direct or mutual gaze is a prerequisite to social interactions. In contrast, when the gaze of someone is averted to another direction than towards oneself, it informs us that we are not the object of interest and that the person is attending to something or someoneelse(Baron-Cohen, 1995; Baron-Cohenetal., 1997b; Emery, 2000) and we then usually turn our attention towards this object.

Averted gaze is frequently used in attention paradigms to demonstrate orienting of attention by gaze (Fig. 1). It has been shown that when a face is centrally presented prior to the onset of a lateral target, target detection is faster when the gaze of the face is directed towards the side where the target later appears and longer when the gaze is looking in the opposite direction (Driver et al., 1999; Friesen and Kingstone, 1998; Hietanen, 1999; Langton and Bruce, 1999; Vuilleumier, 2002; for a very detailed review, see Frischen et al., 2007). This robust effect is seen as early as 3 months of age (Farroni et al., 2000; Hood et al., 1998) and is present even for simple schematic drawings of faces (Friesen and Kingstone, 1998). The fact that this effect is fast (within 200 ms of gaze shift) and occurs when gaze direction is not predictive or even counter-predictive of target location, has been interpreted as reflecting an automatic, reflexive and stimulus-driven (exogenous) orienting of attention mechanism which is impossible to suppress (Driver et al., 1999; Friesen and Kingstone, 1998; Hietanen, 1999; Langton and Bruce, 1999; Mansfield et al., 2003; Ricciardelli et al., 2002).

The magnitude of this orienting effect is similar regardless of the identity of the cue, whether it is a human face, an animal face (e.g. ape or tiger), an object such as an apple or a glove with eyes (Quadflieg et al., 2004), or an inverted face (Tipples, 2005). This suggests that any stimulus possessing eye-like attributes can trigger spatial orienting of attention (but see Langton and Bruce, 1999, for different results with face inversion). Although some studies have reported a similar orientation of attention by simple arrows (Ristic et al., 2002; Tipples, 2002), arguing against the uniqueness of gaze in triggering the orienting effect, a direct comparison of a glove with eyes or with arrows in place of the eyes showed overall faster reaction times to the target for the glove-with-eyes, suggesting gaze cues may exert quantitative rather than qualitative effects on spatial attention, and may reflect different underlying neural networks (Friesen et al., 2004; Quadflieg et al., 2004; but see Hietanen et al., 2006 for faster response times for arrows than gaze cues). Friesen and colleagues further showed that arrows produced only a volitional orienting effect rather than an automatic shift of attention as seen with gaze, further supporting a different neural mechanism for orienting attention by gaze or arrow cues (Friesen et al., 2004). Gaze orienting is also accompanied by small ocular saccades executed in the direction signaled by gaze rather than by the target (Mansfield et al., 2003; Ricciardelli et al., 2002). In an ecological and evolutionist perspective, such an automatic mechanism allows for the fast detection of a potential danger and is thus crucial for species survival.

Some have suggested that the apparent reflexive nature of this effect could be due to the paradigm used, which is always a Posner-like attentional cuing task (Posner, 1987). For instance, one study measured subjects’ eye movements in addition to their response times in two different non-cuing tasks involving the same front-view and 3/4-view faces with averted and direct gaze (Itier et al., 2007c). A purely reflexive orienting mechanism to gaze would predict that, even in these non-cuing tasks, subjects’ gaze would always be attracted by the eye region of the face and would go in the direction signaled by the perceived gaze as shown by eye movement monitoring studies (Mansfield et al., 2003; Ricciardelli et al., 2002). In contrast, the first saccade made by subjects after face presentation was directed to the eye region in 90% of trials when the task required an explicit gaze direction judgment, but in only 50% of trials when the task was a face orientation discrimination, reflecting task modulations of the attraction of attention to the eye region (Itier et al., 2007c). Moreover, these first saccades were performed in the direction signaled by gaze only in the gaze judgment, while they were performed in the direction signaled by head in the head orientation task. This study suggests that orientation of attention towards gaze direction can be modulated by task demands and hence may not be a truly reflexive mechanism (Itier et al., 2007c). The importance of these top-down modulations has recently been demonstrated in a study involving Western Caucasian and East Asian subjects. While the Caucasians fixated more on the internal features of faces and especially on the eyes (Section 2), the East Asian participants tended to look in the center of the faces (Blais et al., 2008). This study demonstrates the effect of culture on face perception and shows that the eye region does not always attract attention. It will be necessary to compare the gaze orienting effects between various populations.

Furthermore, most gaze orienting studies used faces in front-view (e.g. Driver et al., 1999; Friesen and Kingstone, 1998; Friesen et al., 2005; Mansfield et al., 2003; Ricciardelli et al., 2002). However, head orientation may modulate the gaze cuing effect just like it modulates the influence of gaze in various categorization judgments (see Section 2). To our knowledge, only one study so far involved 3/4-view faces in a gaze cuing paradigm (Hietanen, 1999) and found no cuing effect for 3/4-view faces gazing at the side target, reinforcing the idea of an interaction between head orientation and gaze direction. The author interpreted these data as reflecting that the attention orienting system was not using information of others’ gaze direction in reference to the observer but rather in reference to the others’ head orientation.

The fact that gaze cuing is modulated by head orientation (Hietanen, 1999) and that the direction of saccades landing in the eye region is modulated by task (Itier et al., 2007c), suggests top-down influences on the gaze orienting effect which may not be truly reflexive. In a natural environment, eyes alone are rarely the unique source of information on others’ direction of attention. Head orientation, body position and pointing behavior are other important directional cues that are widely used to identify the source of interest of a peer when the eyes are not visible (Emery, 2000; Langton et al., 2000). It would make sense that we could orient towards these cues in a voluntary way depending on the information we need to extract in a given situation rather than in a reflexive way. This idea has been recently supported by a series of attention cuing studies using conflicting simultaneous gaze and arrow cues while subjects attended to one cue and ignored the other (distractor). Nummenmaa and Hietanen (in press) hence showed that gaze and arrows produced similar distracting effects. In other words, gaze distractors did not exert a greater influence on orienting of attention than arrow distractors, arguing against a real automatic mechanism for gaze. Rather, the authors suggested that the attentional systems processed conflicting directional information in a flexible manner. Altogether, these studies argue against the purely reflexive nature of attention orienting by eye gaze, a topic we return to in Section 4.1 on Neuropsychology.

Gaze cuing paradigms have also used direct-gaze rather than averted-gaze stimuli. When the gaze of the face is direct, longer reaction times for target detection have been reported (Senju and Hasegawa, 2005), which are even longer than when gaze is directed to the opposite direction as the target (Vuilleumier, 2002). By attracting attention, direct gaze would increase the disengagement time from the central face before orienting attention to the target location. This attention grabbing effect of direct gaze is also supported by visual search paradigms where staring eyes embedded in an array of averted-eyes stimuli are detected better and faster than averted eyes in arrays of staring eyes (Doi and Ueda, 2007; Senju, 2005; von Grünau and Anston, 1995). This “stare-in-a-crowd-effect” was confirmed in a recent study using more naturalistic stimuli, but only when the faces from which the eye regions were extracted were in 3/4-view (Conty et al., 2006). The use of photographs rather than schematic drawings of eye regions (as in von Grünau and Anston, 1995) is likely the main reason for this difference with previous studies. Overall, these studies support the idea of a specific role of mutual gaze in human cognition.

Finally, these cuing paradigms have also been used with emotional faces. In anxious individuals the orienting-to-gaze behavior is even more pronounced for stimuli representing potential threat such as angry (Holmes et al., 2006) or fearful faces (Fox et al., 2007; Mathews et al., 2003; Putman et al., 2006; Tipples, 2006) compared to neutral ones. In contrast to previous studies which manipulated only two expressions at a time, Fox et al. (2007) used a design where neutral, angry, fearful and happy expressions were intermixed and showed that, in anxious individuals only, the gaze cuing effect was greatly enhanced for fearful faces but also reduced for angry faces compared to neutral and happy expressions. Moreover, the magnitude of the effect was correlated with the anxiety level, positively for fear expression and negatively for angry expression. This study suggests a specific interaction of averted gaze and fear expression rather than with negative emotions in general, and supports the threat-related ecological interpretation of this effect (Fox et al., 2007). For non-anxious individuals, most studies failed to report modulations of the gaze orienting effect with facial expressions (Fox et al., 2007; Hietanen and Leppänen, 2003; Holmes et al., 2006; Mathews et al., 2003; Tipples, 2006). Using dynamic emotional faces (short movies rather than static pictures), one study reported an increased gaze cuing effect for fearful expressions in non-anxious individuals (Putman et al., 2006), and the difference with previous studies may be due to the use of more ecologically valid (dynamic) stimuli. Whether increased cuing effects could also be seen in normal individuals with anger or surprise dynamic expressions, which were not tested in Putman et al. (2006), will need to be addressed in a paradigm including all facial expressions, as used by Fox et al. (2007). Finally, given the important role of the eye region in facial expressions, future studies will have to determine whether the effects of facial expression on target detection in gaze cuing paradigms are due to the expression per se or rather to low-level ocular cues such as the amount of visible white sclera or the size of the eye openness which vary with a given expression (Section 2).

3.2. Gaze perception, joint attention and theory of mind

As we already mentioned, the direction of someone else’s gaze typically signifies where his or her attention is being directed. In a triadic relationship involving two persons (A and B) and one object, the gaze direction of B will inform A of his attention onto the object and A will also attend to it. This is called joint attention as both persons attend to the same object (Fig. 2). However, only one of them uses the other’s gaze direction to orient to the same target (A sees that B looks at the object and A then looks at the object). Shared attention in contrast, implies that both individuals are aware of each other’s object of attention and each of them will use the other’s gaze direction to check that both attend to the same target (A sees that B looks at the object and will attend to the object; B notices that A attends to the object too and A and B look at each other’s eyes – mutual gaze – to make sure they both attend to the same object). Shared attention is thus more complex than joint attention and both play fundamental roles in social cognition.

Gaze following has been reported as early as 3–6 months of age (D’Entremont et al., 1997), although the exact age at which this capacity emerges is controversial (Emery, 2000). Before 9 months, infants can follow their mother’s gaze but are not capable of directing their attention towards the object of her interest. The joint attention capacity, which includes not only gaze monitoring but also pointing gestures, emerges around 9–14 months (Baron-Cohen et al., 1997a) but it’s only around 18 months of age that infants can attend to the same object of interest as their mother if the object is situated outside of their own visual field such as behind them (Butterworth, 1991). Joint attention is very important for the acquisition of language, which starts with the association between a word and the object it represents. Being able to orient one’s attention in the direction of gaze of the person naming the object is thus crucial (Baldwin, 1993; Baron-Cohen et al., 1997a). If someone says “dog” while looking at a dog, a young child listening for the first time to this word will orient his attention in the gaze direction of that person and will associate the word to its meaning. This learning strategy based on the use of people’s gaze direction emerges between 12 and 19 months of age (Baron-Cohen et al., 1997a) and positive correlations between gaze-following at 10–11 months of age and subsequent vocabulary scores at 18 months have been shown (Brooks and Meltzoff, 2005). A recent modeling study showed that gaze-following behavior at 10–11 months of age significantly predicted accelerated vocabulary growth until 2 years of age, even after controlling for the effects of age and maternal education (Brooks and Meltzoff, 2008).

Because of the important role of gaze early in development, the existence of an innate module specialized in the detection of gaze direction (eye direction detector—EDD) has been proposed (Baron-Cohen, 1995). The first function of EDD would be to detect any eye-like stimulus while its second function would be to determine whether the observed gaze is directed towards oneself or elsewhere. This module would play an essential role in the development of shared attention and in theory of mind (ToM). The term ToM refers to the capacity to explain others’ behaviors in terms of mental states, i.e. intentions, desires and beliefs and was originally introduced by primatologists to describe the possibility that chimpanzees understood certain mental states in other chimpanzees (Premack and Woodruff, 1978). This capacity was called a ‘theory’ because it is impossible to directly access others’ minds; we are simply guessing and inferring their mental states. ToM skills emerge around 4–5 years of age (Mitchell and Lacohée, 1991) and can be underlined by cartoon tests portraying social situations in which understanding false belief is essential, as in the classic Sally-and-Ann test (Wimmer and Perner, 1983; for more details on ToM, see Brune and Brune-Cohrs, 2006).

Baron-Cohen proposed the existence of another module called the intentionality detector (ID) which would understand any movement in the environment in terms of volitional movement, i.e. the goal-directed movement of an external agent (Baron-Cohen, 1995). Both EDD and ID would contribute to the development of shared attention, itself necessary for the development of ToM. Perrett and Emery (1994) proposed a direction-of-attention-detector (DAD) module that could process not only gaze cues but any attentional cue including head and body orientation (Perrett and Emery, 1994). They also proposed a mutual attention mechanism and suggested that the activation of EDD or DAD would be necessary for joint attention while shared attention would require the activation of the mutual attention mechanism in addition to EDD or DAD. The fact that theory of mind exists in congenitally blind individuals suggests that vision is not necessary for ToM to develop. However, in the course of normal development, the face and especially the eyes remain one of the richest sources of social information for the attribution of mental states to others. For instance, a 4-year-old child is capable of inferring that someone is thinking about something when their eyes are directed upward and to nothing in particular (Baron-Cohen, 1995). In Baron-Cohen’s theory, gaze direction is thus an important and privileged stimulus for the attribution of mental states. A test based on photographs of isolated eye regions has even been developed for the evaluation of ToM capacities in adults (Baron-Cohen et al., 1997b, 2001a) and in children (Baron-Cohen et al., 2001b). In this force-choice test, subjects need to designate, amongst four words evoking a mental state (e.g. preoccupied, puzzled, reassuring, jealous), which one best describes the eye region presented. This test was called the ‘reading the mind in the eyes’ test and was found appropriate in revealing ToM impairments in special clinical populations such as autistic spectrum disorders (see Section 5). The processing of gaze is thus an extremely important step in developing a social cognition and a theory of mind and relies on a very rich neural network that we describe in the following section.

4.1. Neuropsychology evidence of eye and gaze processing impairments

In humans, a few rare lesion cases have revealed the involvement of cortical and subcortical brain areas in processing the eye region and gaze direction. One patient (MJ), whose extended lesion involved almost the entire right superior temporal gyrus (STG), presented important difficulties in gaze contact (Akiyama et al., 2006a). Her perception of others’ gaze direction was also altered and she perceived left averted gaze as direct, and to a lesser extent, direct gaze as averted to the right. Furthermore, she did not present the normal gaze cuing effect seen in controls (Section 3.1) whereas, like them, she could normally orient to the direction signaled by arrows (Akiyama et al., 2006b). Similarly as what is seen in monkey studies in which the bilateral ablation of some parts of the superior temporal sulcus (STS) considerably impairs the perception of gaze direction (Campbell et al., 1990; Heywood and Cowey, 1992), this neuropsychological case suggests involvement of the right STG in gaze perception and spatial orientation of attention by gaze. MJ presented a sort of deficit in discriminating contra-lesional gaze direction only, suggesting a directionality role of the STG in gaze processing. However, the extent of MJ’s lesion spanning the entire gyrus length precludes any conclusion as to the existence of possible sub-regions of the STG involved more specifically in one or the other aspect of gaze processing. The directionality role of the STG also needs to be confirmed by left lateralized STG lesions (see Akiyama et al., 2006a for a detailed discussion).

A few patients with amygdala lesions also present significant deficits in gaze processing (Young et al., 1995) and in attending to the eye region (Adolphs et al., 2005; Spezio et al., 2007). However, this type of patient usually presents other impairments especially in emotional recognition of fear (Adolphs et al., 1994; Calder et al., 1996) and more generally in the perception of social communication (Adolphs et al., 1998). Interestingly, in the monkey, amygdala or temporal damage also lead to the disruption of social reactions, known as the Klüver-Bucy Syndrome (Klüver and Bucy, 1939). Single cell studies in monkeys have shown an important response of the amygdala to the presentation of faces and isolated eyes (Leonard et al., 1985) and specific nodes of this structure contain cells sensitive to gaze direction (Brothers, 1990; Brothers et al., 1990). A recent monkey fMRI study showed a different activation within the amygdala for facial expressions and for gaze direction that seems linked to attention and arousal (Hoffman et al., 2007). A similar functional and spatial division in the human amygdala could explain the deficits in both facial expressions and gaze processing seen in some amygdala patients. The precise role of the amygdala in gaze processing in humans is still unclear but a recent study showed that the complete ablation of this structure lead to a significant reduction in direct gaze contact during normal conversations with others, along with an abnormal increase of eye movements directed at the mouth rather than the eyes (Spezio et al., 2007). This suggests that the amygdala is involved in directing attention to the eyes, an idea supported by the study of patient SM. Following bilateral amygdala lesions, SM could not recognize the expression of fear (Adolphs et al., 1994) and recently was found to fixate less on the eye region than control participants (Adolphs et al., 2005). Using an image classification technique, the authors showed that, unlike controls, SM was incapable of extracting the relevant information from the eye region necessary to the recognition of fear expression, but remained capable of recognizing fearful expressions when she was explicitly asked to pay attention to the eyes. Although SM is impaired at recognizing fear emotion from other sources such as music (Gosselin et al., 2007), Adolph et al.’s data suggest that her impairment in recognizing fear from a face comes from a problem in spontaneously orienting attention to the eye region and from an incapacity to extract relevant information from that region, rather than from the incapacity to recognize the facial expression per se (Adolphs et al., 2005). This lack of spontaneous exploration of the eye region is consequential to the amygdala lesions but not necessarily to the impairment in recognizing fear itself. However, it is not clear whether SM is also impaired at processing gaze or identity and whether her abnormal use of the eye region could also be seen in other tasks than fear recognition. The role of the amygdala in detecting and orienting attention towards relevant social stimuli like gaze has recently been supported by the case study of five unilateral amygdala-damaged patients who did not present the normal gaze-orienting effect but oriented normally to arrow cues in cuing paradigms (Akiyama et al., 2007).

The abnormal exploration of the eye region has also been reported in a case of prosopagnosia, the inability to recognize familiar faces (Bodamer, 1947). Patient PS is a case of acquired prosopagnosia with no visual agnosia, resulting from bilateral lesions to parts of the occipital and temporal lobes (Rossion et al., 2003a). Contrary to normal participants, PS used the mouth more than the eyes in a face recognition task and this impairment in exploring the eye region was not seen in real social interactions during which she could make normal eye contact (Caldara et al., 2005). PS is also not impaired at recognizing facial expressions or at discriminating gaze direction. The authors suggested that one possible cause of PS’ prosopagnosia could be an impairment in extracting from the eye region the relevant configural information necessary to create accurate face representations in memory, rather than being a general deficiency in attending to the eyes (Caldara et al., 2005). As noted previously, the eyes seem to be used even more so in the processing of familiar than unfamiliar faces (Althoff and Cohen, 1999) and the role of eyes in identity recognition may increase with the familiarity the subject has with the face/person, a topic that will need to be explored in the future.

Impairments in gaze processing have also been reported in a few prosopagnosic patients (Campbell et al., 1990; Perrett et al., 1988), but the exploration of the eye region was not analyzed in these cases. Patient RB was seriously impaired at discriminating whether a face was looking at him or away, even at 20° of gaze deviation (Perrett et al., 1988). Campbell and collaborators reported the cases of an acquired prosopagnosic patient (AP) with a right posterior lesion and a developmental prosopagnosic patient (DP) with no known lesion, both impaired in a task requiring to point at the face whose gaze was directed at them. While the AP was slightly impaired, the DP performed at chance level and a detailed analysis revealed that she was using only face orientation to respond although she was clearly instructed to focus on gaze direction. Interestingly, both patients also presented impairments in processing facial expressions. However, the majority of reported cases of congenital prosopagnosia (i.e. with no lesions) do not present impairments in gaze direction processing (Dobel et al., 2007) and such impairments may not be a characteristic feature of this condition.

Abnormalities in processing both the eye region and its gaze have also been reported in Capgras delusion, an extremely rare neurological syndrome in which the patient believes that very familiar people (e.g. parents, siblings, spouse) have been replaced by identical-looking impostors or robots (Capgras and Reboul-Lachaux, 1923). Patient DS showed spared facial expression discrimination and identity recognition but poor accuracy in judging gaze direction regardless of the degree of gaze deviation used, answering that the face was looking at him even when it was not (Hirstein and Ramachandran, 1997). The authors suggested that his delusion was the result of a disconnection between face sensitive areas (such as inferior-temporal and STS areas) and the limbic system, likely the amygdala (Hirstein and Ramachandran, 1997). This could explain his gaze processing abnormalities given the implication of the amygdala in gaze processing and the influence of face familiarity on gaze perception (Teske, 1988, Section 2.2). However, no MRI data could confirm this disconnection and DS originally suffered a right parietal fracture sustained in a traffic accident (Hirstein and Ramachandran, 1997). The gaze abnormalities could thus be due to this parietal injury as well as, or in place of, the hypothetical disconnection between the limbic system and the face sensitive regions. However, the study of patients with unilateral neglect, a neurological disorder generally arising after right parietal lesions in which patients fail to detect or respond to contralesional stimuli (see Danckert and Ferber, 2006 for a review), does not support the involvement of parietal regions in gaze processing. Indeed, in these patients gaze cues could orient attention to the contralesional field and alleviate extinction, whereas endogenous orienting by arrow cues failed to produce similar effects (Vuilleumier, 2002). If anything, lesions to the right parietal areas were beneficial to gaze processing in these patients. In addition to supporting a specific role of gaze in orienting attention, these data on neglect syndrome support the hypothesis of a disconnection between the limbic system and the face sensitive regions in Capgras patient DS rather than an effect of the parietal injury.

More recently, a case of Capgras delusion with no known psychiatric cause or brain lesion, was reported to spend significantly less time than controls fixating on the eye region of faces, but gaze direction was not assessed in this study (Brighetti et al., 2007). The Skin conductance response (SCR), a measure of emotional reaction and arousal, was also positively correlated with the time spent fixating the eye region of familiar faces in healthy controls (Brighetti et al., 2007). It thus seems that the eye region of very familiar persons such as family members triggers an emotional reaction that seems to be part of the entire familiarity experience. In parallel to the classic ventral route involved in identity processing, some authors have suggested a dorsal route for the affective response to faces (e.g. Bauer, 1984). Eyes could be at the crossroads of these two routes, conveying both configural cues and emotional cues necessary for accurate recognition of highly familiar faces. As reviewed in Section 2, the elements extracted from the eye region are different depending on the task performed: the ocular cues extracted for identity are mainly configural (e.g. distance between the eyes or eyebrows, etc.) and are not necessarily the same as the ones extracted for emotional processing (e.g. size of the white sclera, amount of eye openness, wrinkles around the eyes, etc.). It is possible that the ventral route is used to extract configural cues while the dorsal one is used to extract the information necessary for emotional processing. Future studies will have to address these questions more precisely.

Finally, frontal lesions have also been linked to impairments in gaze attention orienting. A recent study reported the case of patient VCR with bilateral orbito-frontal (OFC) lesions who did not present the normal target detection facilitation seen in gaze cuing paradigms (Vecera and Rizzo, 2006). VCR could normally orient attention towards peripheral cues but was unable to orient attention from word or gaze cues regardless of whether they were predictive (Vecera and Rizzo, 2004) or non-predictive (Vecera and Rizzo, 2006) of target location. According to dual-process theories of attention (Corbetta and Shulman, 2002), peripheral cues tap into exogenous, stimulus-driven attention orienting which is reflexive while symbolic cues such as words and arrows, tap into endogenous or goal-directed attention, which is voluntary. The fact that VCR presented intact exogenous orienting yet could not orient to gaze questions the reflexive nature of gaze orienting that we already mentioned in Section 3.2. Vecera and Rizzo (2006) suggested that, like arrows, gaze orienting was an endogenous, goal-oriented rather than exogenous and reflexive mechanism. In their “associative hypothesis”, they proposed that the apparent reflexive nature of gaze orienting is due to an over-learned association between gaze direction and the location the eyes refer to. The OFC would be responsible for this association and when damaged, no more gaze orienting would be seen. In a developmental perspective, this theory implies that early on, children would learn to associate others’ gaze direction with their object of attention and that this mechanism would involve the development of frontal areas. This idea agrees with the current state of knowledge concerning the role of the joint and shared attention mechanisms in cognitive development (Section 3), and with the hypothesis that the development of ToM skills is related to the maturation of the frontal lobes (Stuss and Anderson, 2004).

The review of the neuropsychological literature suggests that gaze perception involves a large brain network including at least the amygdala, the STG/STS, some ventro-temporal regions involved in face recognition such as the fusiform gyrus (the lesion of which is involved in most types of acquired prosopagnosia), and some frontal areas. In addition, most patients as described above present a specific deficit in the perception of direct/mutual gaze rather than general gaze direction impairments. We now turn to the neuroimaging literature on eye and gaze processing.

4.2. Neuroimaging data of eye and gaze processing

4.3. Temporal aspect of eye and gaze processing: electrophysiology and magneto-encephalography evidence

4.4. Is there an eye detector in the human brain?

The hypothesis of the eye direction detector (EDD) proposed by Baron-Cohen (Section 3.2) assumes that a mechanism in the human brain is specialized for the detection of eyes in the environment, and this “module” would likely be present at birth. Its primary goal would be to detect the presence of eyes or eye-like stimuli in the environment and then to determine whether these eyes are looking towards or away from oneself. However, its existence is not yet supported empirically at the brain level and it is still unknown whether eyes are processed independently in specialized areas or as part of the face in the same face responsive areas. Only a few imaging studies have explored the brain response to isolated eyes and reported conflicting results (Section 4.2.1). On the other hand, ERP data suggest an earlier sensitivity to eyes than faces in young children as well as the existence of different neuronal populations responding to eyes and faces as seen in monkeys, all of which support the possible existence of EDD. The eye detector hypothesis is also supported by behavioral data in infants and by the fact that the scanning of the face always starts with the eyes early on in visual processing (Section 2). If EDD exists, how does it work at the neural level?

In contrast to the classic view derived from the imaging literature, according to which face components are processed first and subsequently integrated into a face percept (Haxby et al., 2000), the EEG/MEG literatures suggest faces are processed first as a whole, and facial features processed later. However, recent studies suggest that holistic processing of the face and featural processing of the eyes occur in parallel and compete depending on the task, attention and context bias (Bentin et al., 2006; Itier et al., 2007b). In most instances holistic processing would prevail over featural processing except in some situations, for instance when viewing isolated eyes (a face with a hood or hat and a scarf up to the nose) or when voluntarily focusing on the eyes for a specific judgment. One recent study combining one image classification technique with EEG recordings suggested that within whole faces, the eyes were attended and processed first, between 100 and 150 ms, i.e. before the N170 peak at which point the whole face is supposedly processed (Schyns et al., 2007). According to the authors “the N170 reflects a process that integrates facial features over time”. However, the use of only one task (emotion discrimination) and one specific image classification technique in that study, precludes any generalization to other tasks and situations. Nevertheless, it agrees with another recent study showing that face discrimination based on face parts is impaired by TMS pulses delivered over the right OFA between 60 and 100 ms after stimulus onset (Pitcher et al., 2007). Although full faces were used, subjects’ judgment was biased towards the eyes and the mouth, suggesting the involvement of the rOFA in facial feature processing before the N170 (and even before, or around, the P1 component). These new results raise once again the empirical question which will have to be addressed by future studies: what comes first, the features or the whole?

Whether a neuronal system specialized in processing the eyes exists in the human brain and is different from the face processing system is thus not yet established. If EDD exists then the traditional models describing the way we process faces (e.g. Bruce and Young, 1986; Haxby et al., 2000) should be revised in order to explain the dynamics of the two systems, because in this view face perception would not be the result of the integration of equally important features within a whole but would rather result from the integration of a system dedicated to the eyes together with a system devoted to processing the rest of the face. If this eye detector mechanism exists, it could (i) recruit one or several brain regions that are also part of the face system, (ii) recruit the entire face system or (iii) recruit another brain network that is different from the face one. Based on the few imaging studies available so far (see Section 4.2), this last possibility is unlikely. It is also possible that eyes recruit the same areas as faces but that these areas are interconnected differently depending on whether the subject focuses on the local or global aspects of the face, or depending on what information is extracted (emotion, identity, intentions, etc.). The EEG/MEG literature suggests the involvement of several brain sources as early as 170 ms after stimulus onset but the exact areas and their differential engagement as a function of task are still debated. In contrast, imaging studies identified several brain nodes involved in the processing of faces and eyes but their temporal dynamics are unclear. The spatio-temporal neural networks involved in these various face and eye processes and their functional connectivity depending on stimulus category and subjects’ tasks remain to be established. Future studies combining EGG/MEG and fMRI techniques, along with network analyses such as structural equation modeling, may help resolve these questions.

We now turn to a pathology in which impairments in processing the eye region and gaze direction may be central to the social interaction deficits observed.

5.1. Impairments in gaze and social cognitive processes in ASDs

Autism is a severe developmental pathology presenting with great impairments in communication and reciprocal social interactions. Autistic individuals have difficulties extracting social cues from faces. Even in Asperger Syndrome (AS) individuals whose IQs remain normal or superior to average, deficits in the recognition of face identity, gender, age and expressions have been reported (Boucher and Lewis, 1992; Celani et al., 1999; Gepner et al., 2001; Tantam et al., 1989; Teunisse and De Gelder, 1994). A recent review however suggests these face processing impairments may not be as important as previously thought (Jemel et al., 2006).

One explanation for these deficits is a lack of interest for the human face (Jemel et al., 2006). Attraction for faces normally starts early during development except for infants later diagnosed with ASDs (Baron-Cohen et al., 1996; Osterling and Dawson, 1994). Autistic children look less at faces than age-matched control children and when they do so, perceptual processes and exploratory ocular movements seem abnormal. In contrast to controls who process faces mainly configurally, ASD individuals rely preferentially on features to process faces (Frith, 1989; Mottron and Burack, 2001; Mottron et al., 2006; Senju et al., 2008). Logically, this should favor the processing of eyes and gaze but eye movement monitoring studies have shown that ocular exploration of faces is disorganized, incoherent and quite variable, and that ASD individuals fixate less on the eye region than controls and more on other parts such as the mouth, the chin, the hair line-forehead limit or the ears (Dalton et al., 2005; Klin et al., 2002; Langdell, 1978; Pelphrey et al., 2002). This is seen during free viewing or when the task requires facial emotion recognition (Pelphrey et al., 2002). A recent study reported that autistic subjects performed ocular saccades away from the eyes, which were correlated with the amount of social information contained in that facial zone (Spezio et al., 2007), supporting the idea of an eye avoidance in autism. These data are in agreement with the clinical diagnosis of the pathology (abnormality in eye contact is one diagnostic criteria of ASDs according to DSM IV, APA, 1994) and suggest an aversion for direct gaze that could explain a lot of non-verbal communication and social interaction impairments these individuals present. A recent study also reported a stronger skin conductance response to direct than averted gaze in children with autism but not in age-matched controls (Kylliäinen and Hietanen, 2006), supporting the idea that mutual contact is too emotionally arousing for autistic individuals who thus need to avoid it.

Furthermore, the spontaneous orienting of attention towards the object of interest indicated by others’ direction of gaze has been reported deficient in autistic children who are insensitive to gaze direction as an index of the speaker’s intention to infer (Baron-Cohen et al., 1997a). This deficit could result from a more general impairment in visual attention orienting but attention cuing studies (Section 3.1) have reported both age matched controls and autistic children presented orienting-to-gaze effects (Kylliäinen and Hietanen, 2004; Ristic et al., 2005; Senju et al., 2004). In one study typically developed children located targets cued by eye gaze more quickly than children with ASDs and targets cued by arrows did not trigger reflexive orienting. In contrast, both social and non-social cues shifted attention to the cued location in children with autism (Senju et al., 2004; see also Vlamings et al., 2005 for similar findings). This abnormal spontaneous orienting towards arrow cues could reveal a lack of preferential social sensitivity to eye cues in autism (Senju et al., 2004). However, caution is required as spontaneous orienting towards the direction of arrows has been found in other studies in normal adults (Ristic et al., 2005; Tipples, 2002) and may thus not be abnormal. Spontaneous orienting towards arrow cues in autism could also reveal a lack of inhibition in autistic children, i.e. a difficulty in disengaging attention (Nation and Penny, 2008). Finally, it has been suggested that rather than being deficient, gaze-orienting might simply be delayed during development in ASDs (see Nation and Penny, 2008 for a more detailed developmental perspective).

When told explicitly to pay attention to gaze, autistic individuals are capable of general knowledge about the eyes and the information they give. For instance, they know that the eyes are necessary for sight. They can discriminate gaze direction (Ristic et al., 2005; Wallace et al., 2006) and identify whether someone is looking at them or what someone is looking at. One study has reported a deficit in detecting direct gaze compared to control children matched for non-verbal development, and while performances in controls were better for direct than averted gaze detection, autistic children did not present any difference between the two types of gaze (Senju et al., 2005). However faces were in 3/4-view in this study and these deficits may thus be due to a processing conflict between head and gaze orientations rather than a real impairment at detecting gaze direction. It was also shown that ASD individuals used specific strategies in gaze orienting: rather than treating the eyes as a social cue like controls, they seem to use mainly low-level information such as pupil direction and the contrast between iris and white sclera (Ristic et al., 2005).

According to some researchers, the social interaction deficits characteristic of ASDs mainly result from ToM deficits (see Section 3) stemming from impairments in processing social information derived from gaze (Baron-Cohen, 1995; Baron-Cohen et al., 1997b, 1999). While normal people understand and infer others’ thoughts effortlessly, ASD individuals present tremendous difficulties in perceiving and understanding others’ mental states (Baron-Cohen, 1995; Baron-Cohen et al., 2001a) and this could be due to abnormal EDD and/or ToM mechanisms. In fact, a complete absence of joint attention (precursor of ToM) at 18 months of age is predictive of a later diagnosis of autism (Baron-Cohen et al., 1996). A recent study reported that the normal development of ToM, measured with animated geometric shapes and language descriptions of mental states, is correlated with the development of the capacity to spontaneously infer the locus of attention of a face using gaze cues in control but not in ASD children (Campbell et al., 2006). This result underscores the links between gaze and ToM and supports the idea of their common neuronal basis as suggested by neuroimaging data (Section 4).

Taken together, studies in ASDs suggest no general impairment in visual attention or specific impairment in gaze orienting per se but rather a deficit in the perception of social cues and a possible impairment in direct/mutual gaze discrimination. The main deficits seem to lie in the incapacity to extract relevant information from the eye region necessary for social communication (Nation and Penny, 2008).

5.2. Neural bases of gaze and social cognition impairments seen in ASDs

Numerous studies have tried to understand the neural basis underlying social deficits in autism and atypical brain activations have been reported in ASD individuals for face perception (Pierce et al., 2001; Schultz et al., 2000), face identity recognition (Dawson et al., 2002) and facial expression discrimination tasks (Critchley et al., 2000). Fewer studies have focused on the neural bases of gaze processing in this pathology. Because of the socio-emotional difficulties characteristic of this disorder, the limbic system has been focused on and anatomical and functional abnormalities of the amygdala have been reported (Abell et al., 1999; Bauman and Kemper, 1985). In a task requiring the attribution of mental states to photographs of isolated eyes, ASD individuals activated prefrontal and superior temporal cortices like controls, but not the amygdala (Baron-Cohen et al., 1999). The amygdala activation is also strongly correlated with the time spent fixating the eyes in ASD individuals (Dalton et al., 2005), supporting the eye avoidance hypothesis.

Recent fMRI studies on autism have also reported neuroanatomical and functional abnormalities in other brain areas, particularly in the temporal lobes (Abell et al., 1999; Carper et al., 2002; Zilbovicius et al., 2006). Using cuing paradigms, two studies showed that whether the gaze of the centrally presented face was directed at a target (congruent gaze) or in the opposite direction (incongruent gaze), a similar STS activation was found in ASD individuals while normal controls presented a larger activation for the incongruent gaze condition (Pelphrey et al., 2003, 2005). This lack of STS modulation to social context confirmed that although a change in gaze direction was detected, its communicative and social value remained impaired in ASDs (see also Mosconi et al., 2005). Hypo-activation of the FG in ASDs has also been consistently reported (Critchley et al., 2000; Dalton et al., 2005; Ogai et al., 2003; Schultz et al., 2000) and could result from a smaller fixation time onto the eyes given the correlation between FG activation and time spent fixating on the eye region (Dalton et al., 2005). Finally, abnormalities in the OFC of autistic individuals have been reported (Schmitz et al., 2007). Although the overall volume of this structure was not different between controls and ASD individuals, it was positively correlated with circumscribed interests in the latter group, a classic symptom of the disorder. In addition, the right lateral OFC was smaller for autistic subjects compared to age-matched controls (Schmitz et al., 2007). Taken together, these structural and functional abnormalities in the amygdala, STS, FG and OFC suggest an abnormal processing of social cues by the entire social brain network in autism.

Just like in brain imaging, abnormalities in ERPs have also been found. Some studies reported a delayed N170 to faces (McPartland et al., 2004; O’Connor et al., 2005) with or without a smaller amplitude of that component in autistic individuals compared to normal controls (Grice et al., 2001; O’Connor et al., 2005). This latency delay was also found for mouths or eye regions presented in isolation (O’Connor et al., 2007). However, the classic latency delay of the N170 for isolated eyes compared to faces (Section 4.3) is found in ASD individuals just like in controls, which suggests a delayed rather than abnormal, early processing of faces and facial features in autism. Recently, normal behavior and ERP components (P1, N170 and P2) were reported in autistic children during both implicit and explicit processing of emotional faces (Wong et al., 2008). However, source analysis suggested the strength and dipole orientations of these components which arose from the visual cortex, the FG and the medial prefrontal cortex, were weaker and/or slower in autism than in typically developed children suggesting abnormal functioning of the underlying neural generators in autism despite similar ERPs on surface. Interestingly, parietal responses were stronger in children with ASDs in that study and could reflect a compensatory analytical strategy to process facial information (Wong et al., 2008).

Neural processing of gaze is also impaired in ASDs. A larger N170 for direct compared to averted gaze has been reported in young autistic children while normal age-matched controls showed no N170 modulation with gaze, just like control adults (Grice et al., 2005). This result could reflect an increased sensitivity to direct gaze very early during visual processing, in agreement with the gaze avoidance hypothesis. Note however, that the N170 in this study was measured at posterior medial sites. At posterior lateral sites where this component is usually maximal, no gaze modulation was seen in any group. This study also involved static stimuli and a passive task. In an explicit gaze direction discrimination task, the change detection in gaze direction was associated with a negative occipito-temporal component (Senju et al., 2005). In normal controls this component was more right lateralized and showed a sensitivity to gaze direction while in ASD children, it showed neither gaze sensitivity nor hemispheric asymmetry. These data suggest an abnormal neural processing of gaze in autism, which is linked to their abnormal perception of social cues and their impairments in ToM.

We have tried to review briefly the impairments in the processing of eyes and gaze in ASDs that seem to be linked to social deficits. We would like to emphasize, however, that most studies focused on high functioning autistic individuals including Asperger Syndrome subjects, rather than on the entire autism spectrum population. One reason for this is that low functioning and severely impaired autistic individuals are extremely hard to test, especially in an MRI scanner or using ERPs. However, scientists should remain careful in the generalization of their findings with these individuals to the entire ASD population. Are face and eye gaze perception related deficits the same in low and high functioning autistic individuals? Or is there a continuum in these impairments across the autism spectrum both behaviorally and at the brain level? For instance, it could be the case that severely impaired subjects simply cannot discriminate gaze direction while AS can but process its social meaning abnormally. An important challenge for future studies will be to bridge the gap between severe autism and Asperger Syndrome using neuroimaging.