Saccades, salience and attention: the role of the lateral intraparietal area in visual behavior

Introduction

We live in a world of sensory overload. Sights, sounds, smells and touches bombard our sensory apparatus constantly, and the primate brain cannot possibly deal with all of them simultaneously. Instead, it chooses the objects most relevant to its behavior for further processing. This act of selection is called attention. William James described attention as “the taking possession by the mind in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought …. It implies withdrawal from some things in order to deal effectively with others … (James, 1890)”. He then described two different kinds of attention: “It is either passive, reflex, non-voluntary, effortless or active and voluntary. In passive immediate sensorial attention the stimulus is a sense-impression, either very intense, voluminous, or sudden … big things, bright things, moving things … blood.” More recently, these two kinds of attention have been described as exogenous and endogenous attention or bottom-up and top-down. Attention is clinically important: the syndrome formerly known as ‘hyperactivity’ is now known as ‘attention deficit and hyperactivity disorder (ADHD).’ Patients with parietal (Critchley, 1949, 1953) and occasionally frontal (Heilman and Valenstein, 1972) deficits show neglect, the inability to attend to a part of the visual field.

The parietal cortex has long thought to be important in the neural mechanisms underlying spatial attention. One parietal area in particular, the lateral intraparietal area (LIP), has been implicated in attentional and oculomotor processes. Although it is clear that LIP has a visual representation, it is not clear whether this visual representation is dedicated to the processing of saccadic eye movements or has a more general attentional function independent of the generation of any specific movement. In this review, we describe three different experiments that examine the role of attention in LIP and its relation to the generation of saccadic eye movements. The first deals with the nature of the visual representation in LIP; the second deals with the independence of LIP activity from saccade planning; and the third with the nature and determinants of visual attention in the monkey.

Behavioral modulation of visual activity in LIP

The neurophysiological basis of attention is not clearly understood, although much evidence suggests that it arises from a gain control on the sensory activity of neurons in various association areas of the brain. This observation, first made in the superior colliculus (Goldberg and Wurtz, 1972), has been extended to a number of other areas in both the dorsal and ventral streams. The standard method for determining the visual properties of a neuron has been, since the development of the fixation task by Wurtz (1969), the response of the neuron to a stimulus that appears suddenly in its receptive field. This definition has a problem. Abruptly appearing stimuli are not only associated with photons exciting rods and cones on the retina, but are also, as James noted, attentional attractors, and the abrupt appearance of a task-irrelevant distractor decreases manual reaction time (Posner, 1980) and increases contrast sensitivity at the site of the distractor (Yantis and Jonides, 1984). Stimuli can enter receptive fields in several ways: one is when a light appears suddenly in the receptive field and a second is when a saccade brings a stable object into the receptive field. Since activity in parietal cortex is associated with attention as well as with vision, the question arises as to whether the ‘visual responses’ of parietal neurons are predominantly visual, i.e. the result of the appearance of photons on the retina, like a retinal ganglion cell, or attentional, the result of extraretinal modulation of the retinal signal.

To distinguish between these alternatives we devised a number of tasks in which the stimulus, rather than appearing de novo in the receptive field, entered the receptive field by virtue of a saccadic eye movement (Gottlieb et al., 1998). This enabled us to stimulate the receptive field using stimuli that did not have the attentional tag of abrupt onset. In these stable array tasks, the monkeys were presented with an array of eight stimuli arranged uniformly in a circular array. These stimuli did not appear or disappear from trial to trial. Instead, they were constant for a block of trials (Fig. 1). The stimuli were roughly 2° in diameter, and varied in shape and color. They were not equated for luminance. They were positioned so that when the monkey fixated the center of the array at least one stimulus appeared in the receptive field of the neuron under study. In the simplest of these tasks the monkey fixated at a position outside the array so that no stimulus was in the receptive field of the neuron being studied, and then, when the saccade brought one of the stable stimuli into the receptive field.

The typical neuron had a brisk response to the sudden appearance of a stimulus in its receptive field during a fixation task (Fig. 2A), and a much smaller response when the same stimulus as a member of the stable array entered the receptive field following the saccade (Fig. 2B). The decrement of response could have been related to the behavioral irrelevance of the stable target, or, it could have been due to a series of other confounds. For example, the movement of the stimulus into the receptive field by the saccade is not exactly the same as its appearance from the flash; the other members of the array might exert some purely visual local inhibition that suppresses the response. To test if these other factors could be responsible for the diminished response to the stable target, we developed the recently flashed stimulus task. In this task, the stable array contained only seven stimuli, but not the one that would be brought into the receptive field by the saccade. This eighth stimulus appeared while the monkey was fixating at the initial position, and remained on for the remainder of the trial. The monkey then made a saccade that brought this recently appeared stimulus into the receptive field. The neuron responded almost as briskly as it did to the abrupt appearance of the stimulus in the receptive field (Fig. 2C, cf. Fig. 2A). Therefore, the difference between the fixation case and the stable target case was not due to the visual or oculomotor differences between the tasks, but to the lack of salience of a stable component of the visual environment. Note that the neuron began to respond at or before the end of the saccade. This was a much lower latency than when the stimulus appeared in the receptive field abruptly (cf. Fig. 2A with Fig. 2C). Presumably this occurred because of the predictive response described previously: neurons in LIP may respond to stimuli that will be brought into their receptive field by saccades earlier than they do to the abrupt appearance of the same stimulus in their receptive fields (Duhamel et al., 1992). The recently appeared stimulus evoked a greater response across the population than did the stable stimulus (using an average response in an interval 200 ms after the end of the saccade: p<0.001 by Wilcoxon signed-rank test; 31 neurons), and evoked a statistically significantly greater response in a majority (23/31, p<0.05 by two-tailed t-test) of single neurons.

The original studies of behavioral modulation emphasized enhanced responses to stimuli that were the subjects of various behavioral tasks (Goldberg and Wurtz, 1972; Robinson et al., 1978; Moran and Desimone, 1985). In this experiment, however, the enhanced activity arises from an intrinsically salient stimulus that has no task relevance: the abruptly appearing stimulus. The intrinsic properties of the stimulus were, perforce, salient. However, it was the sudden appearance of the stimulus in the environment, and not merely the sudden appearance of the stimulus in the receptive field that endowed the stimulus with salience.

Salience does not arise only from intrinsic properties of the stimulus. Stable objects can become important by virtue of their relevance to current behavior, and under these circumstances a member of a stable array can evoke a response from a neuron in LIP. We can show this using the stable target task, a more complicated version of the stable array task (Fig. 3). In this task, the monkey fixated so that the stimulus was not in the receptive field, and a cue appeared during the first fixation. This cue matched one of the symbols in the stable array. The fixation point then jumped to the center of the array and the monkey tracked it with a saccade. Finally, when the fixation point disappeared, the monkey made a saccade to the member of the array that matched the cue. The target indicated by the cue was randomly chosen on each trial among the members of the array. Neurons responded strongly to stable stimuli brought into their receptive field if these were designated as the target of the next saccade (Fig. 4A). The neuron discharged from the first saccade to the second. In contrast, if the identical stable stimulus entered the receptive field but was not designated as the saccade target (the monkey was instructed to saccade elsewhere), neurons responded minimally (Fig. 4B).

It is possible that LIP neurons respond in the stable target task because the monkey is planning a purposive saccade, and the activity is less related to the salience of the stable target than it is to the processes underlying saccade planning. To see how much activity in LIP can be allocated to the planning and generation of a saccade itself we used a task in which the monkey had to make a saccade to a spatial location that had no visual stimulus (the ‘black hole’ task). This task began as a simple version of the stable target task (Fig. 5A). The monkey fixated a point in the center of the array, a cue appeared outside the receptive field matching the symbol in the receptive field, and the monkey made a saccade to that symbol. This trial was repeated a number of times, with the cue always matching the symbol in the receptive field. Then the symbol in the receptive field was removed, leaving a gap, the ‘black hole’ in the array. The monkey, nonetheless, had learned to make a saccade to the center of the hole in the array. Although the cell in Fig. 5 had a robust response when there was a stimulus in the receptive field, it failed to respond when the monkey made the same saccade when the stimulus was absent. Many LIP neurons active in the stable target task frequently failed to respond in this task, and no neurons responded more in this task than in the stable target task when target was in the receptive field. These results illustrated that the visual activity of neurons in LIP are under behaviorally modulated control, and suggested that this visual activity is related to the salience, or attention-worthiness of the stimuli that drove it.

The effect of saccade planning on the visual response of LIP neurons

The stable target and stable array task show that LIP neurons respond to the sorts of stimuli that usually attract attention, but much less to stable stimuli that usually do not attract attention. If LIP were primarily to have a saccade-planning function, then neurons in LIP should filter out irrelevant stimuli that are not the targets for saccades. Once the monkey was committed to a given saccade a stimulus appearing far away from the saccade goal should evoke a weaker response than a stimulus at the saccade goal. When we tested this hypothesis, we found the contrary result: neurons in LIP gave enhanced responses to task-irrelevant distractors that appeared away from the goal of a memory-guided saccade (Powell and Goldberg, 2000). In this experiment we first studied neurons in the delayed saccade task (Fig. 6A), and only chose those that had delay period and/or pre-saccadic activity (Figs. 7A and B). The neuron illustrated in Figs. 7A and B had a visual response, a striking delay period response but a lesser pre-saccadic increase.

We then studied 27 neurons with visual and delay and/or presaccadic activity in the distractor task. In this task the monkey (Figs. 6B and C) performs a delayed saccade task, but 300 ms before the fixation disappeared we flashed a distractor identical to the saccade target for 200 ms. The distractor could appear at the saccade goal (Fig. 6B) or away from it (Fig. 6C). The saccade goal could be in the receptive field or away from it. The second point was symmetric across the vertical and horizontal meridians from the receptive field, and could appear in the same or the opposite hemifield. We randomly interleaved six types of trials: simple memory-guided delayed saccade tasks to the receptive field or to a second point that was outside the receptive field; trials with memory-guided saccades to the receptive field and distractor either in the receptive field or away; and trials with memory-guided saccades away from the receptive field and the distractor either in the receptive field or at the saccade goal.

We found that when a stimulus flashed in the receptive field of an LIP neuron while the monkey was planning a memory-guided saccade away from the receptive field, not only was the response of the neuron not suppressed, it was also slightly enhanced relative to the case when the stimulus appeared at the saccade goal. Figs. 7C and D illustrate this for the same neuron whose activity in the memory-guided saccade task was shown in Figs. 7A and B. This was true across the population and for individual neurons as well. Fig. 8 shows the population response in this experiment, comparing the activity of each cell when to a stimulus appearing away from the saccade goal — i.e. the saccade was made away from the receptive field (ordinate) to a stimulus appearing at the saccade goal (abscissa). If one looked at the activity of the neurons in the 50 ms before the beginning of the saccade, there was no difference between the case when the monkey made a saccade to the receptive field and the distractor was elsewhere, and the case when the monkey made a saccade away from the receptive field and the distractor was in the receptive field. Although the distractor evoked significant activity before the saccade, it had no effect on any measure of saccadic performance: velocity, reaction time, accuracy, or even the early trajectory. Thus LIP, which does filter out responses from behaviorally irrelevant stable stimuli, does not filter out responses from salient stimuli that are known, a priori, not to be saccade goals.

It is well known that in humans there is an attentional advantage at the goal of a saccade (Kowler et al., 1995; Deubel and Schneider, 1996). There is also an attentional advantage at the site of an abruptly appearing stimulus (Yantis and Jonides, 1984). One logical way of interpreting our demonstration of an enhanced response to a stimulus that appears away from the goal of a planned memory-guided saccade is that the activity represents an attentionally salient object in the visual field. Both the saccade goal and the distractor should evoke attention, the former for endogenous reasons and the latter for exogenous reasons. The problem with the results described up to this point, however, is that we did not explicitly measure attention.

The psychophysics of attention in the monkey

In a subsequent study we actually measured attention, and were not only able to correlate the activity of neurons with the monkeys’ attention but make predictions about the monkeys’ performance on the basis of the temporal properties of the neuronal activity (Bisley and Goldberg, 2003). Three methods have been used to describe the locus of attention: a post hoc method which says that if a subject responds to a stimulus it must have attended to it (Goldberg and Wurtz, 1972); a reaction time method, defining the locus of spatial attention as the area of the visual field in which the response to a discriminandum has the lowest latency (Posner, 1980; Bowman et al., 1993); and a contrast sensitivity method, which defines the spatial locus of attention as the area of the visual field with enhanced visual sensitivity (Bashinski and Bacharach, 1980). We chose to use the latter, since it allows us to examine how attention changes over time and under different visual conditions. In addition, it allows us to rule out the possibility that any attentional advantage may be on the motor side of the response, a problem present when defining attention by changes in reaction time.

Our first problem was to establish that in the monkey, like in the humans, attention is pinned at the goal of a saccade. Our task had two components: the monkeys had to plan a saccade to a remembered location and later had to discriminate the orientation of a probe stimulus. On any given day we used four possible saccade targets, which were symmetric across the horizontal and vertical meridians. The probe stimulus consisted of three circle distractors and a Landolt ring whose gap could be on the right or on the left. The orientation of the Landolt ring instructed the monkey either to cancel (gap on right) or to execute (gap on left) the planned saccade when the fixation spot disappeared. We used the contrast sensitivity of the probe to measure attention, by comparing the sensitivity at different spatial locations during the task. We measured the animal’s GO/NOGO discrimination performance at a number of contrasts and calculated the contrast threshold, which we defined as the contrast at which the animal could correctly discriminate the probe in 75% of the trials. The animal’s performance was better when the probe appeared at the saccade goal than when it appeared elsewhere (Fig. 9A).

Because the performance was not statistically different at each of the three locations away from the saccade goal (mean normalized threshold±SEM: 1.03±0.05, 1.02±0.06 and 0.95± 0.03 for monkey B; 1.01±0.08, 1.04±0.08 and 0.95±0.06 for monkey I), we pooled the data from these locations (hollow circles in Fig. 10A). To enable us to compare the thresholds and enhancements measured on different days, we normalized the data from each day, although all statistical comparisons were performed on the prenormalized data. The normalizing factor for each delay from each session was the threshold from trials in which the probe was not at the saccade goal (i.e. the threshold from the right-hand function in Fig. 10A). The attentional advantage at the saccade goal, illustrated by the enhanced sensitivity, was significant throughout the task (p<0.05 by paired t-test): we used stimulus onset asynchronies (SOAs) from the saccade target to probe of 800, 1300 and 1800 ms, and found enhanced performance in both animals for all SOAs that we studied. Fig. 10B shows the normalized contrast thresholds for each monkey at each SOA. In keeping with the human studies we define the attentional advantage as this lowering of threshold at the saccade goal. We assume that the equally high thresholds for the probe at other locations represent the monkey’s performance at loci to which attention has not been allocated by the endogenous process of the saccade plan.

We then had to establish that the monkey’s attention could be drawn to the spatial location of an abruptly appearing distractor. We flashed a task-irrelevant distractor during the delay to see if it could draw attention away from the goal of the planned saccade (Fig. 9B). The distractor appeared on half of the trials and was presented either at the saccade goal or opposite the saccade goal (as in Fig. 9B). The distractor was identical to the target in size, brightness and duration, but appeared 500 ms after the target. It remained on the screen for a duration of 100 ms.

When the distractor appeared in the opposite location of the target, and the probe appeared 200 ms after the distractor, the perceptual threshold went down to the attentionally advantaged level at the site of the distractor (Fig. 11, open points) and rose to the baseline level at the saccade goal (Fig. 11, filled points). However, 700 ms after the distractor had appeared, performance was once again enhanced at the saccade goal and not at the distractor location, and this was also the case with the 1200 ms SOA in monkey I, with a trend towards that result in monkey B. Thus, as in humans the abrupt onset of a distractor in the visual field draws attention. In the monkey this occurs even while the animal is planning a saccade, but the attentional effect of the distractor lasts for less than 700 ms, by which time attention has returned to the saccade goal. The distractor and the saccade plan had the same effect on the monkey’s attention, lowering the contrast sensitivity threshold by the same amount.

The activity of LIP in the attention task

Having established that a saccade plan and a flashed distractor could attract attention in the monkey, we were then able to assess how neuronal activity in LIP correlated with the monkey’s attentional performance. We found that the activity of LIP neurons in the 100 ms interval before the appearance of the probe predicted the monkey’s attention to the site of the probe (Bisley and Goldberg, 2003). We recorded the activity of 41 neurons in LIP with peripheral receptive fields in two hemispheres of the two monkeys from whom we gathered the psychophysical data. All neurons were visually responsive, and the majority had delay period or perisaccadic activity as well. Because we were interested in the neuronal response to the saccade plan or distractor, and because these were constant throughout a given experiment (as opposed to the probe, whose contrast changed from trial to trial), we usually recorded cell activity only with the probe at the highest contrast; for four cells we also recorded activity throughout the range of probe contrasts. Figs. 12A and B show the response of a single neuron, measured at threshold, during the trials in which the target appeared in the receptive field of the neuron while the distractor flashed elsewhere (blue trace) and during the trials in which the distractor appeared in the receptive field and the target elsewhere (red trace). There was no difference between the responses to the distractor and during the saccade plan delay activity measured at threshold or suprathreshold probe contrasts. In this analysis we were not interested in the response to the probe itself, and the illustration shows trials with the longest interval between distractor and probe. This interval is so long that the response to the probe is not present in the time window of the illustration. This neuron displays the typical visual and memory responses of an LIP neuron. In particular, it replicates the earlier finding described above that the events underlying the generation of a memory-guided saccade do not suppress the response to a task-irrelevant distractor away from the saccade goal.

Rather than trying to relate the response of a single LIP neuron to the locus of attention, we normalized the responses of all the neurons by the mean value of all the points for each trial type and then calculated the average normalized activity for each animal (Figs. 12C and D). Presented like this, the data represent a population response to two different events: the appearance of the target and the subsequent generation of the memory-guided saccade; and the appearance of the distractor. Although we recorded the response of each of our neurons to those two events, one could as easily reinterpret the activity as that simultaneously seen in two different populations of neurons with the same overall properties, one with receptive fields at the saccade goal (the ‘target population’ in blue) and the other with receptive fields at the distractor site (the ‘distractor population’ in red).

The activity of the neuronal population in LIP parallels the attentional performance of the monkey. There is a consistent relationship between the activity in LIP (Figs. 12E and F; lower plot) and the behavioral results from the three SOAs measured previously (Figs. 12E and F; triangles). At any given time throughout the trial the attentionally advantaged part of the visual field was that which lay in the receptive fields of the most active neurons. For example, 200 ms after the appearance of the distractor the greatest activity in LIP was in the distractor population (red traces), and the attentional advantage lay at the distractor site. Then 500 ms later the target population again had the greatest activity (blue traces), and attention had returned to the saccade goal.

The appearance of the distractor outside of the receptive field had no significant effect on the delay period activity in the target population. The distractor evoked a brief transient response, which decayed rapidly and soon crossed the level of activity in the target population. We were curious if this crossing point had behavioral significance; if the site with greatest activity in LIP showed the locus of attention, then what happens when the activity at both sites was transiently equal. To determine the period in which there was no single significant preponderance of activity in LIP, we compared the activity at the two sites in a 100 ms window that slid across the entire period in 5 ms steps (Figs. 12E and F; black traces). For each monkey there was a window of 80–90 ms in which there was no significant difference between the activity evoked by the distractor and the activity related to the saccade plan (p>0.05 by Wilcoxon signed-rank test). We hypothesized that during this window of neuronal ambiguity, when activity at both sites did not differ, the monkey would not show a psychophysical attentional advantage at either site.

To test this hypothesis we then went back and measured the contrast sensitivities at the saccade goal and the distractor site at the crossing point in each monkey (455 ms for monkey B and 340 ms for monkey I) and 500 ms later. These times were the center of the window of neuronal ambiguity for each monkey. The top sections of Figs. 12E and F show the behavioral data from the original sessions (triangles) and from the sessions recorded after the physiological experiments (circles). At the crossing point we found no spatial region of enhanced sensitivity in either monkey, but within 500 ms attention had shifted back to the site of the target in both monkeys, with normalized thresholds similar to those seen in the earlier experiment. This is in stark contrast to the effect on saccades: when there was equal activity in the two populations, even in the 50 ms epoch immediately before the saccade, there was no measurable effect on the planned saccade (Powell and Goldberg, 2000). It is possible that there is a period of time following the distractor when attention is shifting, and this period just happens to coincide with the change in activity, while not being related to it. On the other hand, if the activity in LIP were related to attention, then we would expect the behavior to be different in the two animals, because the windows of neuronal ambiguity did not overlap between the two monkeys (see the troughs of the black traces in Figs. 12E and F). We presented the probe to monkey I at the crossing point for monkey B (455 ms) and to monkey B at the crossing point for monkey I (340 ms). We found that the location of the attentional advantage in monkey I had already returned to the saccade goal at the crossing point for monkey B, and that for monkey B the attentional advantage was still at the location of the distractor at the crossing point for monkey I. These data support the hypothesis that there is indeed a correlation between activity in LIP and the locus of attention. Note that both the attentional performance of the monkey and the neuronal properties of LIP were extraordinarily stable. In each monkey we ran the psychophysical experiments for several months after the monkeys learned the task. We then recorded neuronal activity for several months. After we had collected enough neurons to determine the crossing point for each monkey, we returned to the psychophysics for another several months. The psychophysics before and after the recording session had the same characteristics as can be seen by comparing, in Figs. 12E and F, the triangles, which were data points collected before the neuronal recording, and the circles, which were data points collected after the neuronal recording. Similarly, the recordings, from roughly 20 cells per monkey, recorded over a period of months, were able to predict how the monkey would behave months after the recordings.

The absolute level of neural activity did not determine the locus of attention. Instead the locus of attention lay at the part of the visual field associated with the greatest neural activity in LIP. Thus, the delay period activity, which determined the locus of attentional advantage when it was the greatest activity in LIP, could not sustain that advantage when it was swamped by the huge transient response to the distractor. Although at times there was only a small difference in the normalized activity of neurons representing the attentionally advantaged and disadvantaged spatial locations, this difference was extraordinarily robust across the population. This is clear from the examples shown in Figs. 12G–I. These plots compare the mean activity of each neuron measured in monkey B when the saccade goal was in the receptive field with the mean activity during the same 100 ms epoch when the distractor was in the receptive field, at three different times during the paradigm: 200 ms after the appearance of the distractor (Fig. 12G); the crossing point (Fig. 12(H); and 650 ms after the appearance of the distractor (Fig. 12I). In our analysis we included all classes of neurons that we encountered, since the major outputs from LIP contain all the classes of neurons found in LIP (Pare and Wurtz, 1997), and have separately illustrated those with (filled circles) and without (open circles) statistically significant differences in their responses (p<0.05 by t-test). Generally those neurons without significant differences in delay activity during the task had no memory activity based on their responses to a regular memory-guided saccade task.

Although we recorded the bulk of the neurons while the monkey performed the task in a supra-threshold fashion, this had no effect on the neural responses. To test this we recorded the activity of four neurons while the animal performed the task with the full range of contrasts to see if there was any difference in the activity of the neurons during these periods in a more demanding situation. We calculated the mean rates of activity over the same epochs in Figs. 12G–I for both the stimulus configurations (i.e. target in receptive field, distractor out, and distractor in receptive field, target out) when the monkey was working with all contrasts and with only the suprathreshold contrast. We found that the activity was the same regardless of the difficulty level — the regression coefficient for a line fitted through the 24 points was 1.04 with a shift of 1.34 spikes per second (R² of 0.91).

There was a relationship between the performance of the animal and the activity in LIP, manifest in the responses during the 100 ms before the probe appeared in correct and incorrect trials for the two stimulus configurations (Figs. 13A, B). We found that the activity evoked by the saccade plan was lower on error trials than on correct trials, but the activity evoked by the distractor was higher on error trials than on correct trials. However, this activity did not vary with probe location. This suggests that while LIP activity predicts the locus of a perceptual advantage in our experiment, it does not predict on a given trial what the monkey will decide. However, when the monkey is performing the task well, even for suprathreshold probes, there is increased activity at the saccade goal and decreased activity at the distractor site. This is not merely arousal, because the response to the distractor is less than it is when the monkey performs poorly. Arousal would be expected to raise all activity levels.

The results we have discussed up to here show that the attention to be paid to a probe flashed for one video frame is predicted by the activity present in LIP at the time that the probe appears. This is in contradistinction to all previous studies of attentional modulation, which have suggested that the enhanced parietal response to an attended object is responsible for the attention to that object (Bushnell et al., 1981; Colby et al., 1996; Cook and Maunsell, 2002). We found, instead, that the responses evoked by the probe itself did not correlate with our measure of attention. When the probe was in the receptive field, the initial on-responses were identical whether the cue dictated GO to the receptive field, GO elsewhere, or NOGO (Fig. 14A shows the responses of a single cell; Fig. 14B shows the mean responses for every cell in the sample). After 100 ms these responses diverge. When the probe signals GO elsewhere, the response falls rapidly (dashed trace in Fig. 14A). When the probe signals GO to the receptive field, the response falls slightly more slowly and resumes the pre-probe delay period level (black trace in Fig. 14A). When the probe signals NOGO and the monkey was planning a saccade to the receptive field, the response falls far less rapidly, as if the stimulus requiring a cancellation of a saccade plan evokes attention longer than one confirming the saccade plan (gray trace in Fig. 14A). Across the sample the response to this cancellation of a saccade plan is significantly greater than the response to the continuation signal both when the saccade plan is to the receptive field (Fig. 14C) and even more so when the saccade plan and its associated attentional advantage is directed away from the receptive field (Fig. 14D). When the response finally falls, however, it falls to the level of the GO-elsewhere response. Remember that on every trial there was either a probe (the Landolt ring) or a complete ring in the receptive field. We found no difference between the response to the GO probe or the ring in trials in which the saccade plan was directed to the receptive field (Fig. 14E), or away from it (p>0.2, Wilcoxon paired signed-rank test), nor was there any difference in the responses to the probe in correct and incorrect trials. However, the enhanced cancellation response was only seen for the actual NOGO probe and not for a ring in the receptive field when the NOGO probe appeared outside of the receptive field (Fig. 14F).

Discussion

We have described three different experiments that illuminate the role of LIP in eye movements and visual attention. In the first, we show that LIP has representation of the visual world from which static behaviorally unimportant objects have been filtered. Two sorts of objects pass through the filter: those which are important to the animal because they are important to its behavior, in our case the goal of a saccade to a stable object in the environment that matches a cue, and an abruptly appearing stimulus which, although irrelevant to the monkey’s task, is the sort of stimulus that evokes attention. These two different sorts of stimuli evoke the two different sorts of attention described by William James as voluntary and involuntary, which have been more recently described as endogenous and exogenous, or top-down and bottom-up. LIP describes both, and it is impossible to discern from the activity in LIP into which category the stimulus that evoked a response falls.

In the second we show that when a monkey plans a saccade, an abruptly appearing stimulus flashed elsewhere in the visual is not filtered out. Instead, it evokes an even greater response than a similar stimulus flashed at the goal of the saccade. These data are not consistent with LIP having a role exclusively dedicated to the planning of saccadic eye movements, but they are consistent with LIPs providing a representation of significant objects in the visual field, acting as a salience map. Of course, the goal of a saccade is usually the object of attention (Yantis and Jonides, 1984, 1996), and in fact Rizzolatti has postulated that attention in the primate is merely a readout of a possible saccade plan (Rizzolatti et al., 1987).

In the third we provide psychophysical evidence that a monkey’s attention can be influenced by a saccade plan and by the appearance of a task-irrelevant distractor elsewhere in the visual field. Activity in LIP parallels the monkey’s psychophysical performance: when one location in the visual field evokes significantly greater activity in LIP, the monkey exhibits a perceptual advantage for objects in that location.

One surprising result of these experiments is that the absolute value of activity in LIP does not necessarily predict the locus of a monkey’s attention. Instead, the locus of attention is determined by the part of the visual field that evokes the greatest activity in LIP at a given time. For example, during the delay period, before the distractor appeared, the activity in the neurons whose receptive field was at the saccade goal was the highest, and the attentional advantage lay at the saccade goal. A few hundred milliseconds after the distractor appeared, however, the locus of attention had shifted to the distractor site. The activity at the saccade goal was unchanged. It is therefore impossible to know in these cases where the monkey’s attention is by merely looking at one neuron or the representation of one part of space in LIP. The attentional decision must be made by looking at all of LIP, and determining the most active area in a winner-take-all manner. This implies that LIP does not make the decision, but rather like a salience map, it determines the behavioral priority of the various parts of the visual field.

The concept of LIP as a salience map provides insight into the debate about the function of LIP in the organization of behavior. Thus, studies indicate that the lateral intraparietal area (LIP) of the monkey has activity correlating with saccadic intention (Gnadt and Andersen, 1988; Barash et al., 1991; Colby et al., 1996; Snyder et al., 1997), visual attention ( Gottlieb et al., 1998; Bisley and Goldberg, 2003), expected value (Sugrue et al., 2004), perceptual (Shadlen and Newsome, 2001) or economic (Platt and Glimcher, 1999) decision making, perceived motion (Assad and Maunsell, 1995; Williams et al., 2003), elapsed time (Leon and Shadlen, 2003) and stimulus shape (Sereno and Maunsell, 1998). LIP also has activity that seems not to correlate with attention (Platt and Glimcher, 1997; Snyder et al., 1997). We have shown here several examples of the dissociation of activity in LIP from a saccade plan: neurons in LIP describe distractors that are behaviorally never the targets for saccades, and they respond more to stimuli canceling a saccade to their receptive fields than they do to stimuli confirming the saccade.

If, however, we consider that LIP describes a salience map of the visual field in an agnostic manner, without specifying how that salience map is used, the seemingly contradictory views described above can be reconciled. The actual function of the salience map depends on the area to which it projects. LIP has a strong projection to two oculomotor areas: the frontal eye field (Andersen et al., 1990) and the superior colliculus (Ferraina et al., 2002). If a saccade is appropriate, the peak of the salience map in LIP can provide targeting information. If a saccade is inappropriate, for example, during the delay period of a memory-guided delayed saccade task, the oculomotor system can ignore LIP. LIP also projects to inferior temporal cortex, the ventral visual stream areas involved in pattern recognition (Baizer et al., 1991). Neurons here have large, bilateral receptive fields including the fovea, and could not be useful for targeting saccades; but attention is critical for visual perception, and neurons in inferior temporal cortex show attentional modulation (Moran and Desimone, 1985). The ventral stream can use the exactly same salience map for attention that the oculomotor system uses for targeting — the salience map in LIP. The answer to the attention versus intention debate as to the function of LIP is ‘both!’

These experiments raise an important caveat: introspectively attention is both divisible and graded. Thus, human subjects can attend to multiple objects at roughly the same time (Pylyshyn and Storm, 1988; Sears and Pylyshyn, 2000) and a hallmark of parietal damage is the inability to do so (Balint, 1995). Attention can also be graded; the more likely an object is to appear at a given location the better the performance it evokes (Ciaramitaro et al., 2001). However, such distributed attentional processes only operate over relatively long periods of time. Over the time period in which we describe an attentional advantage, in one video frame, there is evidence that attention may be indivisible and ungraded (Joseph and Optican, 1996). However, the activity in LIP is graded, and so over a longer period of time the multiple peaks of the salience map in LIP may contribute to the divisibility and gradation of activity that is present in psychological studies.