Skip to main content
NCBI home page
The Journal of Neuroscience logoLink to The Journal of Neuroscience
. 2004 Jul 14;24(28):6371–6382. doi: 10.1523/JNEUROSCI.0569-04.2004

Target Selection in Area V4 during a Multidimensional Visual Search Task

Tadashi Ogawa 1, Hidehiko Komatsu 1
PMCID: PMC6729540  PMID: 15254093

Abstract

Natural scenes typically contain multiple objects that are unique in different stimulus dimensions so that an object with feature contrast to surrounding objects draws attention and pops out. Furthermore, if we have previous knowledge about the dimension in which a target object differs from the surrounding objects, we will attend to that dimension and more easily detect the target. Our aims here were to elucidate neural mechanisms underlying this type of attention by recording neuronal activities from area V4 and to investigate how visual signals encoding feature contrast between objects are modulated by attention specific to a particular dimension. To accomplish this, we trained monkeys to do a multidimensional visual search task in which two singleton stimuli, unique in the color or shape dimension, were presented with four other identical stimuli. The monkeys had to search for the singleton stimulus that was unique in the instructed dimension while the search dimension was switched between shape and color. We found that individual V4 neurons carry visual signals encoding feature contrast in either shape or color, and this signal is modulated depending on the search dimension. Population responses to the target singleton stimulus were significantly higher than to others, regardless of the search dimension. In most V4 neurons, however, significant response increases occurred only when one particular singleton stimulus was the target. These findings suggest that interaction between bottom-up signals encoding feature contrast between stimuli and top-down signals encoding search dimension occurs in V4 and facilitates adaptive selection of targets in a complex visual environment.

Keywords: attention, V4, feature contrast, visual selection, monkey, saccade

Introduction

Natural scenes typically consist of a large number of objects situated against a cluttered background so that feature contrast between the objects plays a key role in their visual selection. That is, an object having features that contrast with surrounding objects pops out and draws attention automatically (Treisman and Gelade, 1980). Neural correlates of this process have been studied extensively using the oddball search task (Schall et al., 1995b; Basso and Wurtz, 1997; Hasegawa et al., 2000; Constantinidis and Steinmetz, 2001; Iba and Sawaguchi, 2002; McPeek and Keller, 2002). A single singleton (target) stimulus can be discriminated using only bottom-up sensory signals encoding feature contrast between objects (Thompson et al., 1997; Murthy et al., 2001). However, when there is more than one singleton object, and they are unique in different dimensions, top-down attention should be needed to select one of the singleton objects as the target. If we have previous knowledge about the dimension in which the relevant object differs from the surrounding objects, we should be able to detect the target more easily by attending to that dimension, even though we do not know its particular shape or color (Bacon and Egeth, 1994; Egeth and Yantis, 1997).

The aim of the present study was to examine the neural mechanisms underlying this type of attention in area V4, where neurons have selectivity for the color and shape of stimuli (Zeki, 1983; Desimone and Schein, 1987; Schein and Desimone, 1990; Gallant et al., 1993; Kobatake and Tanaka, 1994; Pasupathy and Connor, 1999), and many are sensitive to the feature contrast between stimuli (Desimone and Schein, 1987; Schein and Desimone, 1990). In addition, lesion studies have shown that area V4 plays an important role in visual selection and discrimination tasks (Schiller and Lee, 1991; Schiller, 1993; De Weerd et al., 1999, 2003), and V4 neurons are influenced by attention to spatial location and to the visual features of objects (Moran and Desimone, 1985; Haenny and Schiller, 1988; Spitzer et al., 1988; Maunsell et al., 1991; Motter, 1993, 1994; Connor et al., 1997; Luck et al., 1997; McAdams and Maunsell, 1999; Reynolds et al., 1999). It was not known, however, whether attention to a particular dimension that specifies feature contrast between objects modulates neuronal responses in area V4. To address that question, we trained monkeys in a multidimensional visual search task (see Fig. 1A) in which a singleton stimulus in a particular dimension served as the target while the relevant search dimension was switched between shape and color. We found that for most V4 neurons, significant attentional influences affected responses to a particular combination of search dimension and singleton type.

Figure 1.

Figure 1.

Open in a new tab

Design of the multidimensional visual search task and behavioral data. A, Two singleton stimuli, one unique in the color dimension, the other unique in the shape dimension, were presented simultaneously with four other identical stimuli. The stimulus in the receptive field (RF, gray circles) had one of three singleton types: shape singleton (black rectangle frames, top row), color singleton (gray frames, bottom row), or nonsingleton (type not shown). Monkeys had to make a saccade (arrows) to one of the singleton stimuli, depending on the search dimension: to the shape singleton stimulus in the shape search (left column) or to the color singleton stimulus in the color search (right column). Examination of the two search conditions was conducted in separate blocks. When the relevant search dimension was switched, the behavioral significance of the two singleton stimuli, target (solid frames) and distractor (dashed frames), were exchanged. B, Incidence of the destinations of saccades in the multidimensional search task. Each plot indicates the mean ± SD values across all the recording sessions of the neurons studied. Open and filled circles represent the color and shape search conditions, respectively. C, Effect of trial repetition on the task performance. Behavioral scores were computed and plotted for every trial in the first 5 trials and every 10 trials in the remaining trials after a switch in search dimension. The scores for trials after 95 repetitions are represented by the rightmost data points. The dashed line indicates the chance level (1/6 = 16.7%).

Parts of this paper have been published previously in abstract form (Ogawa and Komatsu, 2002).

Materials and Methods

Subjects and surgery. Two monkeys (Macaca fuscata), weighting 5.0-6.0 kg, were used in this study. In preparation for electrophysiological recording, an eye coil, a head holder, and a recording chamber were implanted during aseptic surgery using standard techniques while the animal was under deep anesthesia (intravenous sodium pentobarbital). Using magnetic resonance imaging before surgery, the recording chamber was placed at the stereotaxic coordinates over the V4 cortex. We also identified the positions of the superior temporal and lunate sulci in each hemisphere at an early stage in the recording sessions to confirm that the recordings were made from area V4 (Gattass et al., 1988). All procedures for animal care and experimental protocols were in accordance with the National Institutes of Health Guide for the Care and Use of Laboratory Animals (1996) and were approved by the Animal Experimentation Committee of the Okazaki National Research Institutes.

Apparatus. The experiments were under the control of three personal computers using custom-written software that presented the stimuli, recorded neural signals and eye positions with sampling rates of 25 kHz and 1 kHz, respectively, and controlled the task schedule. Monkeys were seated in a primate chair during the experiment. Eye position was monitored using the scleral search coil technique (Fuchs and Robinson, 1966; Judge et al., 1980). Visual stimuli were presented on a video monitor (120 Hz noninterlace, 800 × 600 resolution) that was viewed binocularly from a distance of 65 cm in a dark room. The background was uniformly dark gray with a luminance of 0.1 cd/m2. The fixation spot was a small spot subtending 0.1°. As described below, the six stimuli in each array had the same luminance (10 cd/m2) and size. The stimulus size was changed depending on its distance from the fixation point (e.g., 0.92°2 at 8° eccentricity).

Behavioral tasks. The monkeys were trained to perform a multidimensional visual search task. In this task, an array of six stimuli appeared on an imaginary circle around the fixation point, all separated by the same distance (Fig. 1 A). In each array, two singleton stimuli, one unique in color (color singleton) and the other in shape (shape singleton), were presented with four other identical (nonsingleton) stimuli. One of the singleton stimuli served as the target and the other a distractor, depending on the ongoing search dimension (the monkeys needed to attend to the feature contrast in the shape or color dimension to detect the relevant singleton stimulus). In the shape search condition (left column), the shape singleton stimulus was the target, and the color singleton stimulus was the distractor; vice versa in the color search condition (right column). A nonsingleton stimulus (condition not shown) never became the target in either search condition. (This stimulus condition will be referred to as nontarget to contrast with the target and distractor conditions.) Monkeys were required to make a saccade to the target. Similar stimulus configurations were used previously in various psychophysical experiments (Theeuwes, 1991; Bacon and Egeth, 1994). Figure 1 A illustrates the four conditions of the receptive field stimulus. In all four cases, the stimulus in the receptive field is the same (an open cross in this example). In the top row, the stimulus in the receptive field is a shape singleton, whereas in the bottom row, it is a color singleton; the nonsingleton stimulus condition is not shown. The stimulus in the receptive field in the top left and bottom right panels is the target, whereas that in the top right and bottom left panels is the distractor. We were thus able to manipulate the relationships between the singleton type (shape singleton or color singleton) and the behavioral significance (target or distractor) of the receptive field stimulus by switching the search dimensions.

Each trial began with the onset of a fixation spot at the center of the monitor screen. The monkeys had to fixate on that spot within a window of ±0.5-0.75°. After 800-1500 msec, the fixation spot was extinguished, and the stimulus array was displayed. When a monkey made a single saccade to the target, it received a liquid reward. Inaccurate saccades were defined as those landing outside a square window around the target whose size was 35% of the diameter of the stimulus array. If the gaze deviated from the fixation window before the stimulus array was presented, or if the saccade latency was too short (<120 msec), the trial was aborted without reward. Identical sets of 13 different visual stimulus arrays were used in both search conditions. Of these, one array was used in a catch trial in which all six stimuli had the same color and shape. In that case, the monkeys were given a reward when they held fixation throughout the trial (data from catch trials were excluded from the present analysis). The remaining 12 stimulus arrays included a combination of two shapes, two colors, and three singleton types (shape singleton, color singleton, or nonsingleton stimuli in the receptive field). The three singleton types were presented with the same frequency at three possible positions: one within the receptive field and two 120° from the receptive field in either the clockwise or counterclockwise direction. At the remaining three positions, nonsingleton stimuli were always presented. Stimulus shape and color were randomly selected from two shapes and two colors determined in advance and were presented with the same frequency at all six possible positions. Consequently, the monkeys could not use bias for a specific shape, color, or position to select the target. With monkey 1, two fixed shapes and colors were used throughout the training and recording experiments. With monkey 2, the shapes and colors were chosen from among five candidate shapes and colors for each neuron so that the responses to two shapes and colors were as different as possible. Shape and color search conditions were sequentially conducted in separate trial blocks (typically 100 trials). Usually, more than two blocks for each of the shape and color search conditions were repeated for each neuron. The average and minimum numbers of blocks were 3.28 and 1, respectively. The stimulus dimension defining the target was instructed by the color of the fixation spot (shape search, red; color search, blue). In most of the neurons studied, the color cue was given only for the first 10 trials of each block, and a white spot was presented in the rest of trials. Data from both cued and noncued trials were used for the present analysis. The search dimension in the initial block was randomly selected.

Electrophysiology. Single-cell activity was recorded from the region in V4 representing the lower visual field (5-10°). Single microelectrodes (Frederick Haer & Co., Bowdoinham, ME) were passed through the dura using a set of grids with 0.5 mm precision. The visual responses and the receptive fields of each neuron were first assessed in a passive visual fixation task. Neurons with receptive fields located within an area extending from 4 to 10° from the fovea and that did not include the fixation point were selected for further testing. We adjusted the radius and orientation of the stimulus array so that only one stimulus was presented in the receptive field of the neuron and confirmed this by examining the response when the stimulus in the receptive field was removed and only the remaining five stimuli were presented. If the response to the five-stimulus array was >20% of the response to the single stimulus in the receptive field, the data from that neuron were excluded from the analysis.

Data analysis. Typically, >10 repetitions of each trial type were performed for each neuron; the minimum accepted for analysis was seven. Only data obtained in correct trials were analyzed. We analyzed neural activity occurring between the shortest visual latency in area V4 (70 msec) (Schmolesky et al., 1998) and the times around the minimum onset latency of the saccades (monkey 1, 250 msec; monkey 2, 180 msec). A few trials in which saccades occurred earlier than the above timing were excluded from the data analysis (monkey 1, 0.9%; monkey 2, 2.0%).

With the present experimental design (Fig. 1), each monkey had to detect the target by combining information about the singleton type and the ongoing search dimension. We therefore focused on those neurons that conveyed signals encoding information about the types of singleton (shape singleton, color singleton, and nonsingleton) of the receptive field stimulus and analyzed how a change in search dimension (shape search and color search) influenced visual responses. To identify neurons that showed differential visual responses to the three types of singleton in the receptive field, we performed three-factor ANOVA using the two shapes, two colors, and three singleton types as main factors for shape and color search separately. Unless otherwise indicated, further analysis was conducted with those neurons in which the main effect of singleton type was significant in at least one search condition (significance was assigned at p < 0.05). Before the ANOVA, a square root transformation of firing rates was performed to convert neural activity with a Poisson distribution to a normal distribution. It should be noted that, because the different singleton types of the receptive field stimulus (shape singleton, color singleton, and nonsingleton) each corresponded to one of the different conditions in the behavioral significance (target, distractor, and nontarget) in each search condition, neurons whose activity was significantly changed in response to the behavioral significance of the receptive field stimulus were also detected in this ANOVA. We distinguished the neuronal modulation correlated with singleton type from that correlated with behavioral significance by comparing the responses recorded in the two search conditions as described in more detail in Results.

To analyze how the effects of singleton type and search dimension modulate the visual responses to stimuli presented in the receptive field, the effects of singleton type and search dimension on neuronal activity were isolated for each neuron by averaging the firing rates across the stimulus features (i.e., the shape and color of the stimulus in the receptive field) in each search condition. As a result of this averaging, we obtained six responses (two search dimensions × three singleton types) that were then compared across the population of recorded neurons. For this comparison, normalized responses were used to compensate for cell-to-cell differences in response magnitude. For each neuron, a set of six normalized responses was computed as the ratio of the response to each singleton type in each search dimension to the average response across all 24 trial conditions (two search dimensions × three singleton types × two shapes × two colors). Hereafter, the normalized responses to the shape singleton, color singleton, and nonsingleton stimulus in the shape search will be referred to as (RS)s, (RC)s, and (RN)s, respectively, whereas those in color search will be referred to as (RS)c, (RC)c, and (RN)c. If there is no difference among the six normalized responses, their values should be 1. Significant differences between normalized responses were determined using permutation test (Efron and Tibshirani, 1993). In each permutation, trial data were randomly shuffled across the responses to two different singleton stimuli. To avoid the mixing of responses to different shapes or colors, the shuffling was separately conducted in individual shape and color. This procedure was iterated to produce 1000 total permutations. In each permutation, values of the normalized responses were computed in the same way as for the actual data, and the two normalized responses were compared. If the magnitude of the difference in the actual data was larger than the permuted differences in >950 iterations, it was deemed that these two responses were significantly different (at the p < 0.05 level).

To quantify the magnitude of the selectivity for the stimulus shape or color, shape and color selectivity indices were calculated as shape selectivity index =|R11 + R12 - R21 - R22|/(R11 + R12 + R21 + R22), and color selectivity index =|R11 - R12 + R21 - R22|/(R11 + R12 + R21 + R22), where Rij (i = 1, 2; j = 1, 2) is the response magnitude for the stimulus with the ith shape and jth color value. A value of 0.33 in each index indicates that one of the shape (or color) stimuli induced twice as strong a response as the other, and a value of 0 indicates there was no difference between the responses to each shape (or color).

To estimate the degree of the sensitivity to the shape singleton or color singleton stimulus, shape and color singleton indices were defined as shape singleton index = ((RS)s + (RS)c - (RN)s - (RN)c)/((RS)s + (RS)c + (RN)s + (RN)c), and color singleton index = ((RC)s + (RC)c - (RN)s - (RN)c)/((RC)s + (RC)c + (RN)s + (RN)c). These indices indicate the degree to which each neuron preferred shape singleton (or color singleton) stimuli over nonsingleton stimuli.

Results

Behavioral performance

The monkeys' behavioral performances during the multidimensional visual search task (Fig. 1A) were well above the chance level (1/6 = 16.7%). Over the course of the recording sessions, the average scores in the shape and color searches were, respectively, 84.6 and 85.8% correct for monkey 1 and 84.0 and 87.8% correct for monkey 2. The performances were thus slightly better in the color search than the shape search in both monkeys. The average saccade reaction times were 363.3 msec in the shape search and 358.3 msec in the color search for monkey 1 and 274.6 and 252.8 msec for monkey 2. Table 1 shows the mean performance scores and reaction times for the two monkeys in six types of trials (three singleton types × two search dimensions). A two-factor ANOVA, using singleton type and search dimension as main factors, revealed a significant (but marginal) difference only between the average reaction times in the two search conditions for monkey 2 (p = 0.048). No significant differences were detected under the other conditions (p > 0.05). That there was little intrasubject variability in the reaction times, and scores obtained under the two search conditions imply that, for each monkey, the difficulty of the task was similar under the two search conditions, even though intersubject differences clearly existed.

Table 1.

Mean performance (percentage of correct trials) and reaction time (in milliseconds) of the two monkeys across recording sessions and across different conditions

Correct responses (%)
Saccade reaction time (msec)
Shape search
Color search
Shape search
Color search
S(T)
C
N
S
C(T)
N
S(T)
C
N
S
C(T)
N
Monkey 1 83.1 85.6 85.1 83.3 86.8 87.3 367.9 369.1 352.9 361.3 358.7 354.9
Monkey 2
85.0
85.0
82.0
86.3
88.2
89.0
273.6
281.6
268.7
255.4
253.2
249.7
Open in a new tab

The receptive field stimulus singleton types are indicated by S (shape singleton), C (color singleton), and N (nonsingleton); (T) indicates that the receptive field stimulus is the target. Differences in the performance scores and in the reaction times were evaluated using a two-factor ANOVA applied to each monkey, with search condition and singleton type as the main factors. A significant difference was defected only in the reaction times for the two search conditions with monkey 2 (p <0.05).

On the whole, the monkeys made errors in 15.9 and 14.2% of trials in the shape and color search tasks, respectively (Fig. 1B). In most error trials, the monkeys made saccades to the distractor or one of the four nontarget stimuli. The percentages of saccades to the distractor stimulus were 9.2% in the shape search and 11.9% in the color search. Those to the nonsingleton stimuli were 6.4% in the shape search and 2.3% in the color search. Other errors (0.2% in the shape search and 0.1% in the color search) included saccades to other locations and saccades with abnormal dynamics. A tendency to make erroneous saccades to the distractor singleton stimulus more often than the nonsingleton stimuli was similar to previously reported results in humans (Theeuwes et al., 1999; Godijn and Theeuwes, 2003) and monkeys (Bichot et al., 2001). Although the distractor in one search dimension was the target in the other, the frequency of the saccades to the distractor was considerably lower than to the target, indicating that the monkeys were readily able to switch search dimensions. To more precisely assess how quickly the monkeys adjusted to a dimensional switch, the percentages of correct responses were separately calculated every trial within the first five trials after a dimensional switch and every 10 trials in the remaining trials (Fig. 1C). The monkeys made more errors (<80% correct) in the first one or two trials after a switch, but their performances were better than or ∼80% in the subsequent trials, indicating rapid adaptation to the new search dimension after a switch.

Visual responses of single neurons in the multidimensional visual search task

We recorded from 132 single visual-responsive neurons in V4 of the two monkeys (n = 101, monkey 1; n = 31, monkey 2). Thirty-five of these neurons were excluded from the analysis because their responses in the control experiment did not fulfill the criterion that only one stimulus was presented in their receptive field (see Materials and Methods). The remaining 97 neurons did fulfill that criterion and were used for further analysis (n = 75, monkey 1; n = 22, monkey 2). Figure 2 shows the distribution of the response magnitudes of these neurons in the control experiment; most showed no response at all or only very weak responses.

Figure 2.

Figure 2.

Open in a new tab

Distribution of response magnitudes in a control experiment that confirmed that one stimulus was presented in the receptive field (RF). The abscissa indicates the response magnitudes to a five-stimulus array normalized by the magnitude of the response to a single stimulus presented inside the receptive field. Response magnitude was defined as the difference between the discharge rates during 300 msec periods before and after the array onset. Negative values indicate that a five-stimulus array evoked inhibitory responses. The ordinate indicates the number of cells. Dark bars represent 36 neurons, the activity of which was significantly modulated depending on the singleton type of receptive field stimulus (see Materials and Methods); open bars represent all neurons.

Figure 3A shows the responses of one example neuron in the multidimensional visual search task. Insets in each panel show the stimulus arrays used. The left column shows responses recorded under the shape search condition, and the right column shows those recorded under the color search condition. To exclude modulation caused by the saccadic eye movements, the spike density functions in Figure 3A, as well as those in all subsequent figures, are constructed only from neural data recorded before the onset of eye movement in each trial. The rows represent the four different features of the stimulus in the receptive field, which was made up of a combination of two colors and two shapes. Note that, in each row, the stimulus in the receptive field is the same. This neuron responded strongly when a blue cross (top row) or a blue circle (third row) was presented. The three curves in each panel represent responses elicited when the color singleton (green line), shape singleton (red line), or non singleton (blue line) stimulus was presented within the receptive field. In each panel of the top and third rows, the responses to color singleton stimulus (green lines) are consistently above those to the shape singleton stimulus (red lines). That the responses to the color singleton stimulus are substantially larger than those to the shape singleton stimulus indicates that the responses were influenced not only by the stimulus in the receptive field but also by those in the surround. Furthermore, careful inspection reveals that the difference between the responses to the color singleton and shape singleton stimulus is larger in the color search (right column) than in the shape search (left column).

Figure 3.

Figure 3.

Open in a new tab

Stimulus configuration and responses of a single V4 neuron recorded from monkey 2. A, The left and right columns represent the shape search and color search conditions, respectively. Different rows represent the four possible features of the stimulus in the receptive field (gray circles) made up of a combination of two colors and two shapes. Insets in each panel show the stimulus arrays used. Red and green frames indicate the presentation of the shape singleton and color singleton stimulus in the receptive filed, respectively. Other formats are the same as in Figure 1 A. Open and filled elements in the stimulus arrays correspond to blue and yellow colors, respectively. Note that, in each row, the stimulus in the receptive field is always the same. Red, green, and blue lines in the spike density functions indicate the responses when the shape singleton, color singleton, or nonsingleton stimulus respectively was presented within the receptive field. The thick solid, thick dashed, and thin dotted lines in the spike density functions respectively indicate that the stimulus in the receptive field was the target, distractor, or nontarget. The responses are temporally aligned at the onset of the stimulus array. The spike density functions were smoothed with a Gaussian function (SD = 10 msec). To exclude the influence of the saccadic eye movements, the spike density functions in this and all subsequent figures were constructed using only neural data recorded before the onset of eye movement. The shaded area in each panel indicates the period of data analysis for monkey 2 (70-180 msec). This neuron preferred blue (first, third rows) over yellow (second, fourth rows) and showed stronger responses to color singleton stimuli (green line) than to shape singleton stimuli (red line). B, Average firing rates elicited by the four possible features (bottom) of the receptive field stimulus recorded in the shape search (left) and color search (right) conditions. Responses to the shape singleton, color singleton, and nonsingleton stimuli are indicated by diamonds, triangles, and circles, respectively. Each plot depicts the mean ± SE values of each response. Dashed lines indicate the spontaneous activity measured in each search condition during a 300 msec period before onset of the stimulus array.

Figure 3B shows the average responses of the same neuron to four different features of the receptive field stimulus during the period from 70 to 180 msec after the onset of the stimulus array (Fig. 3A, gray rectangles). We evaluated the significance of the effects of the task variables (singleton type, shape, and color of the receptive field stimulus) on neuronal responses with three-factor ANOVA, using the shape, color, and singleton type as main factors, for the shape search and color search conditions separately (see Materials and Methods). For the responses shown in Figure 3, the effect of the color factor was significant under both search conditions (p < 0.001), but the effect of the shape factor was significant only in the color search (p = 0.022). The effect of singleton type was significant in the color search (p = 0.002) but not in the shape search (p = 0.07).

The effects of singleton type and search dimension can be clearly seen in the summed responses of the same neuron (Fig. 4A). In this case, the responses were averaged across stimulus features (two shapes and two colors) so that only the effects of singleton type and search dimension are seen (see Materials and Methods). First, this neuron was most strongly activated when the stimulus in its receptive field was singleton in the color dimension (green lines). Even when the color singleton stimulus served as the distractor in the shape search, it still induced the greatest activity (left column). This suggests that the observed neuronal modulation was mainly determined by the singleton type (color singleton) of the receptive field stimulus. Second, the response of this neuron was influenced by the search dimension. The response to the color singleton stimulus was stronger in the color search than in the shape search, even though identical stimulus arrays were used in both search conditions.

Figure 4.

Figure 4.

Open in a new tab

Summed responses of the same V4 neuron shown in Figure 3. A, The responses to the four stimulus features shown in Figure 3A were averaged to see the effects of receptive field stimulus singleton type and of search dimension. Conventions are the same as in Figure 3A. B, Normalized responses to RS, RN, and RC recorded in the shape search (left) and color search (right) (see Materials and Methods). Error bars indicate 1 SD. #Significant differences (permutation analysis, p < 0.05).

To quantify the response modulation in the summed responses (Fig. 4A), we respectively computed the normalized responses to the shape singleton, color singleton, and nonsingleton stimuli in the shape search [(RS)s, (RC)s, and (RN)s] and in the color search [(RS)c, (RC)c, and (RN)c] (see Materials and Methods). Figure 4B shows that the normalized responses to the color singleton stimulus (RC) were significantly larger than those to other stimuli (RS and RN) in both the shape search and color search conditions (permutation analysis, p < 0.05) (see Materials and Methods), which confirms that the responses were dependent on the singleton type. That the normalized response to the color singleton stimulus (RC) was significantly bigger in the color search than in the shape search (permutation analysis, p < 0.05) confirmed the effect of the search dimension. On the other hand, the responses to the shape singleton stimulus (RS) did not differ under the two search conditions (permutation analysis, p > 0.05), suggesting that the search dimension had no significant effect on the response to the shape singleton stimulus. That the effect of the search dimension was exerted specifically on a particular type of singleton stimulus suggests an interaction between the effects of the search dimension and the singleton type.

Summarized in Figure 5 is a similar analysis of the activity of an example neuron recorded in the other monkey (monkey 1). In this case, ANOVA showed significant effects of both color and shape on the neuronal responses (three-factor ANOVA, main effect of shape or color, p < 0.001) (Fig. 5A). The effect of singleton type was significant in the shape search (three-factor ANOVA, main effect of the singleton type, p = 0.03), but not in the color search (p = 0.66). The response to the shape singleton stimulus (B, red lines, C, diamonds) was significantly stronger than that to the other stimuli in the shape search (permutation analysis, p < 0.05) (Fig. 5C), but there was no significant difference in the responses in the color search (permutation analysis, p > 0.05). This neuron thus also exhibits an interaction between the influences of the search dimension and the singleton type of the receptive field stimulus. Indeed, the degree of the interaction in this neuron appears to be even stronger than in the cell shown in Figure 3.

Figure 5.

Figure 5.

Open in a new tab

Responses of another V4 neuron recorded from monkey 1. A, Average responses to the four different features of the receptive field (RF) stimulus. Conventions are the same as in Figure 3B. B, Summed response calculated by averaging the neural responses across the four stimulus features. The shaded areas indicate the data analysis period for monkey 1 (70-250 msec). Conventions are the same as in Figure 4 A. C, Normalized responses. Conventions are the same as in Figure 4 B.

Population responses

To identify neurons that showed significant differences in their responses across the three singleton types of receptive field stimulus, we conducted a three-factor ANOVA for all 97 neurons studied. In 36 neurons (37%; n = 26, monkey 1; n = 10, monkey 2), the main effect of the singleton type was significant (p < 0.05) in at least one search condition (Table 2). Of those, 25 were significant in the shape search, and 24 were significant in the color search; 13 were significant in both search conditions, and 23 were significant in only one search condition. The neurons whose responses are shown in Figures 4 and 5 are examples of these neurons. The neuron in Figure 4 showed response modulation for only the color singleton stimulus, and that in Figure 5 showed modulation for only the shape singleton stimulus. Likewise, 23 neurons showed response modulation for only one of color or shape singleton stimuli, and 11 neurons showed response modulation for both color and shape singleton stimuli (permutation analysis, p < 0.05). The remaining two neurons did not exhibit a significant difference between the responses to either of the singleton stimuli and nonsingleton stimulus. Numbers of neurons showing significant dependence on the shape and color of the receptive field stimulus are also summarized in Table 2. Relationships between the sensitivity to shape or color and singleton type will be examined later (see Fig. 9).

Table 2.

Numbers of neurons showing significant dependence on the receptive field shape, color, or singleton type

Factor of main effects
Shape/color
Search
Singleton type
Shape and color
Shape
Color
Nonselective
Shape search 25 21 5 9 1
Color search
24
20
6
8
2
Open in a new tab

A total of 97 neurons were examined. Significance of the effect on neural activity was evaluated by three-factor ANOVA using shape, color, and singleton type of the receptive field stimulus as factors (see Materials and Methods). Values of p < 0.05 were considered significant.

Figure 9.

Figure 9.

Open in a new tab

Relationships between the sensitivity to shape or color and shape or color singleton. A, The shape selectivity index of each neuron is plotted against the shape singleton index. The solid oblique line is the linear regression line. B-D, Color selectivity versus shape singleton indices (B), shape selectivity versus color singleton indices (C), and color selectivity versus color singleton indices (D). Conventions are the same as in A.

In addition to these 36 neurons, 15 neurons (15%; n = 11, monkey 1; n = 4, monkey 2) showed a pure interaction between singleton type and other factors (shape or color). Of these, nine were significant in the shape search, eight in the color search, and two in both. Results of the population analyses described below are essentially the same whether these 15 neurons are included. In the following population analyses, we examined the properties of the former 36 neurons (unless otherwise indicated) because responses of these neurons more reliably signaled information about singleton type.

We then used these 36 neurons to analyze the effects of singleton type and search dimension on neuronal activity in more detail. Figure 6 shows the distributions of the magnitudes of the normalized responses to each of the three singleton types recorded under the shape search and color search conditions. In the shape search (left column), the distribution of response magnitudes for the shape singleton stimulus (Fig. 6A) was shifted to larger values (mean, 1.059) compared with the distributions of the other stimuli (Fig. 6B, color singleton mean, 0.976, C, nonsingleton mean, 0.941; Wilcoxon signed rank test, p < 0.05). In the color search (right column), in contrast, the distribution of response magnitudes for the color singleton stimulus (Fig. 6E, mean, 1.065) was, on average, significantly larger than the distributions for the other stimuli (Fig. 6D, shape singleton mean, 1.008, F, nonsingleton, mean, 0.95; Wilcoxon signed rank test, p < 0.05). Because the shape singleton stimulus in the shape search and the color singleton stimulus in the color search were both target stimuli, the above results indicate that the average population response was significantly stronger when the stimulus in the receptive field was the target. Furthermore, the response magnitude for the distractor stimulus was larger than that for the nonsingleton stimulus in both search conditions (Fig. 6B vs C, D vs F; Wilcoxon signed rank test, p < 0.05), even though the distractor stimulus was irrelevant in the search task. This means that, regardless of whether the stimulus was the target or the distractor, singleton stimuli evoked, on average, larger responses than nonsingleton stimuli. We evaluated the timing of modulation in normalized peristimulus time histograms by using a sliding time window of 40 msec width with a 10 msec step. As a population, the target stimulus was significantly discriminated (Wilcoxon signed rank test, p < 0.05) from the nonsingleton stimulus at 110 msec (shape search) or 130 msec (color search) and from the distractor stimulus at 190 msec (shape search) or 200 msec (color search).

Figure 6.

Figure 6.

Open in a new tab

Distributions of the normalized responses of 36 neurons showing significant dependence on the singleton type. A-C, Responses to the three singleton types (RS, RC, RN) obtained in the shape search condition are illustrated. Arrows indicate the mean values in the distribution for each condition. D-F, Distribution of the normalized responses obtained in the color search condition. *Significant differences between the mean values in the two corresponding distributions (Wilcoxon signed ranks test, p < 0.05); n.s., not significant.

When we compared the responses obtained in the two search conditions, a significant difference was observed in the responses to the shape singleton (Fig. 6A vs D) and color singleton (Fig. 6B vs E) stimuli (Wilcoxon signed rank test, p < 0.05), and the response was stronger when the receptive field stimulus was the target than when it was the distractor, which is consistent with the results of the comparison of the responses within each of the two search conditions described above. There was no significant difference (p > 0.05) in the responses to the nonsingleton stimuli under the two search conditions (Fig. 6C vs F). Thus, responses were significantly influenced by the search dimension, making the population response stronger when a given singleton stimulus was the target than when the same singleton stimulus was the distractor. At the population level, therefore, whether the receptive field stimulus was the target or the distractor appears to be an important factor.

Dependence of neural activity on sensory and behavioral contexts

We have seen that in the responses of individual neurons, neural modulation can be strongly influenced by the singleton type of receptive field stimulus (Fig. 4). At the population level, moreover, the responses of V4 neurons were significantly stronger when a given singleton stimulus was the target than when the same singleton stimulus was the distractor (Fig. 6). This suggests that both sensory and behavioral components of the receptive field stimulus are involved in the neural modulation. So, to understand the functional significance of the modulation, we sought to distinguish between these two components. The sensory component is reflected by neural modulation that correlates with the types of feature contrast between the stimulus in the receptive field and neighboring stimuli outside the receptive field (shape singleton or color singleton). We will use the term sensory context to indicate this component. The behavioral component is reflected by neural modulation that correlates with whether the receptive field stimulus is the target or the distractor. We will use the term behavioral context to indicate this component.

We evaluated the relative importance of the sensory and behavioral contexts in the 36 neurons analyzed by computing the difference between the normalized responses to the shape singleton and color singleton stimuli (RS and RC) and comparing the difference values obtained under the shape search and color search conditions. Figure 7 shows the results of this analysis. The abscissa and ordinate indicate the difference values (RS - RC) obtained in the shape search and color search, respectively, while each data point represents a pair of difference values obtained from one neuron. A positive (negative) value on either axis indicates that the shape singleton stimulus elicited a stronger (weaker) response than the color singleton stimulus. If the sensory context is more important, neurons should maintain a preference for one of the shape singleton or color singleton stimuli across the two search conditions, and there should be a positive correlation between the difference values, with the data points clustering along a diagonal line with a positive slope (y = x). On the other hand, if the behavioral context is more important, there should be a negative correlation, and the data points should cluster along another diagonal line with a negative slope (y = -x). Figure 7 indicates that there was a significant positive correlation in the distribution of the data points (correlation coefficient, r = 0.71; p < 0.001), which clearly shows that the sensory context acted as the dominant factor across the recorded neurons, and V4 neurons tend to maintain their preference for the sensory context (shape singleton or color singleton) across both search conditions. However, an effect of the behavioral context can also be seen. The ellipse in Figure 7 indicates the 95% confidence interval for the data plots. The center of the distribution (gray cross) significantly deviated from 0 in the right bottom direction (Wilcoxon signed rank test, p < 0.05). This reflects the fact that the mean of the response magnitudes for the shape singleton stimulus (RS, target condition) was larger than that for the color singleton stimulus (RC, distractor condition) in the shape search and opposite in the color search (Fig. 6). When the 15 cells with pure interaction between singleton type and features (shape or color) were included in the analysis of Figure 7 (n = 51), the results were essentially the same as described above; there was a significant positive correlation in the distribution of the data points (correlation coefficient, r = 0.74; p < 0.001) as well as the shift of the center of the distribution in the right-lower direction (Wilcoxon signed rank test, p < 0.05).

Figure 7.

Figure 7.

Open in a new tab

Dependence of neural modulation on sensory and behavioral contexts. In the scatterplot, differences in the normalized responses to the shape and color singleton stimuli (RS - RC) obtained in each of the two search conditions are compared for each neuron. Each data point represents a pair of difference values obtained from one neuron. A positive (negative) value on each axis indicates that the shape singleton stimulus elicited a stronger (weaker) response than the color singleton stimulus. There is a significant positive correlation between these two values (r = 0.71; p < 0.001). The ellipse and solid line indicate the 95% confidence interval for the data points and the main axis of the ellipse, respectively. Bar graphs at the top and right depict the distributions of values along the horizontal and vertical axes, respectively. Arrows indicate the means of these distributions. *The center of the two-dimensional distribution (gray cross) significantly deviated from zero in the right bottom direction (Wilcoxon signed ranks test, p < 0.05). Filled circles indicate neurons in which response modulation was significantly different from zero (permutation analysis, p < 0.05).

The results in Figure 7 were also confirmed by the analysis at the level of responses to the individual stimulus feature. We have computed the difference between the responses to the shape singleton and color singleton stimuli using the responses to individual stimulus features (two shapes × two colors). Of 144 cases (36 neurons × 4 feature conditions), 47 (30 cells) were significantly differed from 0 (permutation analysis, p < 0.05). We did the same analysis as in Figure 7 to these 47 cases and found that there was a significant positive correlation between the difference values obtained in two search conditions (correlation coefficient, r = 0.67; p < 0.001). This indicates that the sensory context acted as the dominant factor at the level of responses to individual stimulus features, too.

Interaction between singleton type and search dimension

We have shown that both the sensory and behavioral contexts are involved in the modulation of visual responses in our multidimensional visual search task (Fig. 7). At the population average, the behavioral context was observed as an increment in the response to the target stimulus. However, as shown in examples of single V4 neurons (Figs. 4, 5), this increment was observed exclusively when a particular singleton stimulus was the target, suggesting an interaction between the search dimension and the singleton type. To determine whether this phenomenon is common across the entire population of the neurons, in each neuron we tested whether the increase in the response to the target stimulus was dependent on the singleton type of the receptive field stimulus. To do that, we separately computed the magnitudes of the increases in the responses to the target stimuli for the two singleton types (color singleton and shape singleton) and compared the two values. The magnitude of the response increase was determined as the difference between the normalized response to a given singleton stimulus when it was the target and the response to the same singleton stimulus when it was the distractor, i.e., (RS)s - (RS)c for the shape singleton stimulus and (RC)c - (RC)s for the color singleton stimulus. Figure 8 shows the results of this analysis. The abscissa indicates (RS)s - (RS)c, and the ordinate indicates (RC)c - (RC)s, while each data point represents a pair of difference values obtained from one neuron. A positive (negative) value on each axis indicates that a given singleton stimulus elicited a stronger (weaker) response when it was the target than when it was the distractor. If the response increment for the target stimulus occurred regardless of singleton type, there should be a positive correlation between the difference values, and the data points should cluster along a diagonal line (y = x) in the right top quadrant. This was not the case, however. Of the 36 neurons, 21 showed a significant response difference for at lease one type of singleton stimulus (permutation analysis, p < 0.05) (Fig. 8, filled symbols). Of those, eight neurons (filled triangles) showed a significant difference in their response to only the shape singleton stimulus, and seven of those showed an increment in the response to the target stimulus; the cell analyzed in Figure 5 is an example of such cells. The remaining neuron showed a response decrement. On the other hand, 10 of the 21 neurons (filled squares) showed a significant difference in their response to only the color singleton stimulus. Of those, nine showed an increment in the response to the target stimulus; the cell analyzed in Figures 3 and 4 is an example of such cells. Again, the remaining neuron showed a response decrement. Only 3 of the 21 neurons (filled circles) showed significant response differences to both shape singleton and color singleton stimuli, and only one of those showed an enhanced response to both singleton stimuli. There is thus a clear tendency for the increment in the response to the target stimulus to depend on the singleton type, which suggests that there is an interaction between the search dimension and the singleton type. When the 15 cells with pure interaction between singleton type and features (shape or color) were included in this analysis (n = 51), the results were essentially the same as described above; 10 and 15 neurons showed a significant difference in their responses only to the shape singleton or to the color singleton stimulus (permutation analysis, p < 0.05), respectively. Only three neurons showed a significant difference in the responses to both singleton stimuli.

Figure 8.

Figure 8.

Open in a new tab

Interaction between singleton type and search dimension. Examined was whether modulation of the response to the target stimulus depends on the singleton type of the receptive field stimulus. The abscissa indicates the difference in the response magnitude when the shape singleton stimulus was the target compared with when it was the distractor; the ordinate indicates the difference for the color singleton stimulus. Filled triangles represent neurons in which there was a significant difference in only the response to the shape singleton stimulus (permutation analysis, p < 0.05); filled squares, those in which there was a significant difference in the response to the color singleton stimulus; and filled circles, those in which there was a significant difference in the responses to both shape singleton and color singleton stimuli. Open circles indicate cells in which there was no significant difference. Data points located above the negatively sloped diagonal line (n = 33) indicate that the magnitude of the response increment was larger than the magnitude of the response decrement (see Results).

The responses of some neurons were increased when one of the singleton stimuli was the target but were diminished when the other singleton stimulus was the target. Nevertheless, in most neurons, the magnitude of response increase was larger than that of the decrease. This is reflected in Figure 8 by the fact that most neurons (n = 33) fell above the diagonal line with a negative slope (y = -x). As a result, the population average of the responses to the target stimuli was stronger than that to the distractor stimuli when the target was either the shape singleton or color singleton stimulus (e.g., Fig. 6).

The results in Figure 8 were also confirmed by the analysis at the level of responses to the individual stimulus feature. The magnitude of the response change when a given singleton stimulus became the target was evaluated for each of four stimulus features (two shapes × two colors). In 44 (25 neurons) of 144 cases (36 neurons × 4 feature conditions), the response change was significantly different from 0 (permutation analysis, p < 0.05). Of those, 19 and 21 cases showed a significant change in their response to only the shape singleton or color singleton stimulus, respectively. Only 4 of the 144 cases showed significant response changes to both shape singleton and color singleton stimuli. Therefore, at the level of responses to individual stimulus feature, there was a clear tendency that the response change between the two search conditions depended on the singleton type of the receptive field stimulus.

Shape and color selectivity

Many V4 neurons exhibit color or shape selectivity or both (Zeki, 1983; Desimone and Schein, 1987; Schein and Desimone, 1990; Gallant et al., 1993; Kobatake and Tanaka, 1994; Pasupathy and Connor, 1999). Given that many of the neurons in the present study exhibited a preference for either the color singleton or shape singleton stimulus, it would be interesting to know whether these two sensitivities are related. In fact, some neurons showed a correspondence in their preference for the two sensitivities. For example, the neuron shown in Figure 3 was color-selective and preferred the color singleton stimulus. To examine the correlation between shape or color selectivity and the degree of the preference for the shape or color singleton stimulus, we computed indices for shape or color selectivity and shape or color singleton (see Materials and Methods). The shape selectivity (color selectivity) index reflects the degree to which a neuron differentiated between two shapes (colors), whereas the shape singleton (color singleton) index reflects the degree to which a neuron differentiated between the shape singleton (color singleton) stimulus and the nonsingleton one. Figure 9 shows that there was no significant correlation between the shape selectivity and shape singleton indices (Fig. 9A; r = 0.29; p > 0.05), the color selectivity and shape singleton indices (Fig. 9B; r = 0.27; p > 0.05), the shape selectivity and color singleton indices (Fig. 9C; r = -0.32; p > 0.05), or the color selectivity and color singleton indices (Fig. 9D; r = 0.31; p > 0.05). In sum, there appears to be no systematic relationship between feature (shape or color) selectivity and sensitivity to the singleton type, although the small number of colors and shapes used in the present experiments make this conclusion tentative.

Discussion

With our experimental design, the shape, color, and position of the target stimulus were randomly changed from trial to trial. Consequently, the only information available was about the feature contrasts that defined the singleton stimulus in a given stimulus array (bottom-up information) and about the search dimension that defined the relevant singleton (target) stimulus (top-down information). Under those conditions, the monkeys could not use selective attention either to specific feature values (e.g., blue circle) or to a specific position to select the target (discussed later). This makes the present task different from visual selection and attention tasks in which the target stimulus is determined on the basis of its features or position (Motter, 1993, 1994; Chelazzi et al., 1998; Bichot and Schall, 1999). The present task also differs from ordinary oddball search tasks, in which a single singleton stimulus serves as the target (Schall et al., 1995b; Basso and Wurtz, 1997; Hasegawa et al., 2000; Constantinidis and Steinmetz, 2001; Iba and Sawaguchi, 2002; McPeek and Keller, 2002). In those tasks, the target stimulus is discriminable using only bottom-up information about feature contrast between stimuli. In our task, because two singleton stimuli exist in each stimulus array, the subject cannot know which one of two singleton stimuli is the target without top-down information about the search dimension. To solve the task, therefore, it should be necessary to combine bottom-up (singleton type) and top-down (search dimension) information. Consistent with that idea, we found that modulation of the visual responses of V4 neurons tended to occur with particular combinations of the search dimension and the singleton type (Fig. 8). Recent studies in area V4 examining the effects of spatial attention on luminance contrast against the background have also shown that bottom-up stimulus contrast signals and top-down attentional influences are closely combined in neural modulation (Reynolds et al., 2000; Martinez-Trujillo and Treue, 2002; Reynolds and Desimone, 2003) (also see Gilbert et al., 2000; Treue, 2003). Our present results suggest that the interaction of signals encoding feature contrast between stimuli and search dimension modulates visual responses in area V4.

Neural modulation by feature contrast to surround stimuli

Previous studies have shown that responses to stimuli in the classical receptive field are modulated by “surround” stimuli outside the receptive filed, and neural responses tend to be stronger when there is feature contrast between the receptive field stimulus and the surround stimuli than when there is no such contrast (Allman et al., 1985; Tanaka et al., 1986; Schein and Desimone, 1990; Knierim and van Essen, 1992; Kastner et al., 1999; Li et al., 2000). The stimulus configuration used in the present study differed from those in earlier studies in that the surround stimuli comprised a relatively small number of independent objects that were separated from the receptive field stimulus by long distances. Nevertheless, effects of the feature contrast between stimuli were still apparent. V4 neurons mainly signaled information about the sensory context (shape singleton or color singleton) (Fig. 7), and the population response to the singleton stimulus was larger than to nonsingleton stimuli, even when the singleton stimulus was the distractor (Fig. 6). Thus, V4 neurons could detect the contrast in shape, color, or both between isolated objects that were spatially segregated.

Previous studies of attention

Previous studies have demonstrated the effects of nonspatial, feature-based attention in extrastriate areas, and attention to a particular feature of the visual stimulus modulated neural responses (Haenny and Schiller, 1988; Spitzer et al., 1988; Maunsell et al., 1991; Motter, 1994; Gottlieb et al., 1998; Treue and Martinez Trujillo, 1999; McAdams and Maunsell, 2000). For example, Motter (1994) showed that the activity of V4 neurons was selectively enhanced when the color or luminance value (e.g., black or blue) of the receptive field stimulus matched what the monkey attended. In contrast, our monkeys had to pay attention to the feature contrast in a particular dimension, not the feature value itself. In this case, attentional effects heavily depended on the singleton type of the receptive field stimulus, even though the feature values of the receptive field stimuli were identical (see Fig. 3).

Previous studies have also demonstrated that spatial attention to a particular location can modulate neural responses (Moran and Desimone, 1985; Motter, 1993; Connor et al., 1997; Luck et al., 1997; McAdams and Maunsell, 1999; Reynolds et al., 1999). In the present study, increments in the responses to target stimuli occurred only when one particular singleton stimulus became the target (Fig. 8). Consequently, the observed attentional modulation cannot be explained simply as the effect of spatial attention. For the same reason, it cannot be explained by saccade-related activities in V4 (Fischer and Boch, 1981a,b). Because, in our experiment, the configuration of the stimulus array itself determined which stimulus was the target, the monkeys could not select the target in a given stimulus array until the singleton stimulus was identified in the relevant dimension. Only then could spatial attention be allocated to the target. We analyzed the neural activity occurring during a relatively early period, 80 or 110 msec earlier than the average saccade latency in each monkey (see Materials and Methods). It may be that neuronal responses that are correlated with the effect of spatial attention emerge at a later time, near the saccade onset.

An attentional effect on the contextual influences of the surround stimulus has been shown in V1 (Ito and Gilbert, 1999), but, although the effects of spatial attention were examined in that study, it remained unknown whether attention to feature contrast affects the contextual influences of the surround stimulus in visual areas, including area V4. The present study revealed that, in most neurons, attention to color or shape contrast can modulate the responses to one particular singleton stimulus (Fig. 8), which suggests that attention signals can modulate the contextual influences of surrounding stimuli in a singleton-type-specific manner.

Psychophysical studies have shown that a singleton with salient features may capture attention in a bottom-up manner, even when that singleton is completely unrelated to the search task (Theeuwes, 1991; Theeuwes and Godljn, 2002). This is in line with the present finding that a singleton stimulus induced more activation than a nonsingleton stimulus, even when that singleton stimulus was irrelevant (distractor) in the ongoing search condition (Fig. 6). This result may also give a neuronal explanation for why the monkeys made erroneous saccades to the distractor singleton stimulus more frequently than to the nonsingleton stimulus (Fig. 1B).

The role of V4 in visual selection

Areas such as the frontal eye field (FEF), lateral intraparietal area (LIP), and superior colliculus (SC) form networks for representing visual saliency and visual selection of the target for saccadic eye movements (Schall et al., 1995b; Basso and Wurtz, 1997; Thompson et al., 1997; Gottlieb et al., 1998; Colby and Goldberg, 1999; Murthy et al., 2001; McPeek and Keller, 2002), and the relevance of visual areas to such saliency networks was recently proposed (Martinez-Trujillo and Treue, 2002; Mazer and Gallant, 2003; Reynolds and Desimone, 2003; Treue, 2003). The results of the present study support the idea that V4 provides important signals for visual selection. We found that V4 visual responses carry combined signals encoding both sensory (shape singleton and color singleton) and behavioral (target or distractor) context. This caused the diversity in the properties of recorded V4 neurons. Thus, V4 can provide not only information about feature contrast but also about behavioral relevance to the areas involved in the visual selection and generation of oculomotor commands.

One important difference between V4 and the aforementioned visuomotor areas is that, in those areas, neurons are not selective with respect to stimulus features such as motion, orientation, and color (Mohler et al., 1973; Pigarev et al., 1979; McPeek and Keller, 2002) (but see Sereno and Maunsell, 1998; Toth and Assad, 2002). Except for a particular condition in which monkeys are trained exclusively on targets of one particular color (Bichot et al. 1996), FEF neurons typically do not exhibit selectivity for stimulus color (Schall et al., 1995b). This raises the question of how the visuomotor areas discriminate the relevant singleton stimulus from other stimuli without information about stimulus features. That question may be explained as follows. We found that the sensory context of the neuronal modulation varied from neuron to neuron; the shape singleton stimulus induced the strongest activity in some neurons, but in others, the color singleton stimulus induced the strongest activity (Fig. 7). At the same time, an increment in the response to the target stimulus was seen across V4 neurons, even though the search dimension in which the response increment occurred differed from cell to cell (Fig. 8). Therefore, if the responses of all V4 neurons are summed, we would expect that the effect of the sensory context would be diminished, but the increment in the response to the target would accumulate. Indeed, the pooled response of V4 neurons reliably discriminates the target (Fig. 6), even though individual neurons clearly signal sensory information (e.g., Figs. 3, 4). The cells downstream of V4 simply need only to sum the signals from V4 neurons to detect a target with feature contrast in the relevant dimension, even though they do not encode the stimulus feature or its contrast.

One area where perhaps such neural pooling might occur is the FEF, which is directly connected to V4 (Schall et al., 1995a). In visual selection, FEF neurons encode the target in a dimension-invariant manner (Schall et al., 1995b), and one recent experiment showed that neural activity in the FEF could influence responses of V4 neurons (Moore and Armstrong, 2003). The effect of search dimension on V4 activity is also consistent with several recent studies providing evidence that FEF activity related to saccade target selection is influenced by memory or attention (Bichot and Schall, 1999; Bichot et al. 2001; Sato et al. 2003). During the course of target selection, therefore, V4 and areas involved in visual selection, such as FEF, LIP, and SC, might dynamically interact with one another (Schmolesky et al., 1998) to accomplish proper selection of the target.

Footnotes

This work was supported by grants-in-aid for scientific research for young scientists (B) and the Project on Neuroinformatics Research in Vision through special coordination funds for promoting science and technology from the Ministry of Education, Science, Sports and Culture of Japan. We thank M. Ito for valuable comments and M. Togawa and I. Yokoi for technical assistance.

Correspondence should be addressed to Dr. Hidehiko Komatsu at the above address. E-mail: komatsu@nips.ac.jp.

Copyright © 2004 Society for Neuroscience 0270-6474/04/246371-12$15.00/0

References

  1. Allman J, Miezin F, McGuinness E (1985) Stimulus specific responses from beyond the classical receptive field: neurophysiological mechanisms for local-global comparisons in visual neurons. Annu Rev Neurosci 8: 407-430. [DOI] [PubMed] [Google Scholar]
  2. Bacon WF, Egeth HE (1994) Overriding stimulus-driven attentional capture. Percept Psychophys 55: 485-496. [DOI] [PubMed] [Google Scholar]
  3. Basso MA, Wurtz RH (1997) Modulation of neuronal activity by target uncertainty. Nature 389: 66-69. [DOI] [PubMed] [Google Scholar]
  4. Bichot NP, Schall JD (1999) Effects of similarity and history on neural mechanisms of visual selection. Nat Neurosci 2: 549-554. [DOI] [PubMed] [Google Scholar]
  5. Bichot NP, Schall JD, Thompson KG (1996) Visual feature selectivity in frontal eye fields induced by experience in mature macaques. Nature 381: 697-699. [DOI] [PubMed] [Google Scholar]
  6. Bichot NP, Chenchal Rao S, Schall JD (2001) Continuous processing in macaque frontal cortex during visual search. Neuropsychologia 39: 972-982. [DOI] [PubMed] [Google Scholar]
  7. Chelazzi L, Duncan J, Miller EK, Desimone R (1998) Responses of neurons in inferior temporal cortex during memory-guided visual search. J Neurophysiol 80: 2918-2940. [DOI] [PubMed] [Google Scholar]
  8. Colby CL, Goldberg ME (1999) Space and attention in parietal cortex. Annu Rev Neurosci 22: 319-349. [DOI] [PubMed] [Google Scholar]
  9. Connor CE, Preddie DC, Gallant JL, Van Essen DC (1997) Spatial attention effects in macaque area V4. J Neurosci 17: 3201-3214. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Constantinidis C, Steinmetz MA (2001) Neuronal responses in area 7a to multiple-stimulus displays: I. Neurons encode the location of the salient stimulus. Cereb Cortex 11: 581-591. [DOI] [PubMed] [Google Scholar]
  11. De Weerd P, Peralta III MR, Desimone R, Ungerleider LG (1999) Loss of attentional stimulus selection after extrastriate cortical lesions in macaques. Nat Neurosci 2: 753-758. [DOI] [PubMed] [Google Scholar]
  12. De Weerd P, Desimone R, Ungerleider LG (2003) Generalized deficits in visual selective attention after V4 and TEO lesions in macaques. Eur J Neurosci 18: 1671-1691. [DOI] [PubMed] [Google Scholar]
  13. Desimone R, Schein SJ (1987) Visual properties of neurons in area V4 of the macaque: sensitivity to stimulus form. J Neurophysiol 57: 835-868. [DOI] [PubMed] [Google Scholar]
  14. Efron B, Tibshirani R (1993) An introduction to the bootstrap. New York: Chapman and Hall.
  15. Egeth HE, Yantis S (1997) Visual attention: control, representation, and time course. Annu Rev Psychol 48: 269-297. [DOI] [PubMed] [Google Scholar]
  16. Fischer B, Boch R (1981a) Selection of visual targets activates prelunate cortical cells in trained rhesus monkey. Exp Brain Res 41: 431-433. [DOI] [PubMed] [Google Scholar]
  17. Fischer B, Boch R (1981b) Enhanced activation of neurons in prelunate cortex before visually guided saccades of trained rhesus monkeys. Exp Brain Res 44: 129-137. [DOI] [PubMed] [Google Scholar]
  18. Fuchs AF, Robinson DA (1966) A method for measuring horizontal and vertical eye movement chronically in the monkey. J Appl Physiol 21: 1068-1070. [DOI] [PubMed] [Google Scholar]
  19. Gallant JL, Braun J, Van Essen DC (1993) Selectivity for polar, hyperbolic, and Cartesian gratings in macaque visual cortex. Science 259: 100-103. [DOI] [PubMed] [Google Scholar]
  20. Gattass R, Sousa AP, Gross CG (1988) Visuotopic organization and extent of V3 and V4 of the macaque. J Neurosci 8: 1831-1845. [DOI] [PMC free article] [PubMed] [Google Scholar]
  21. Gilbert C, Ito M, Kapadia M, Westheimer G (2000) Interactions between attention, context and learning in primary visual cortex. Vision Res 40: 1217-1226. [DOI] [PubMed] [Google Scholar]
  22. Godijn R, Theeuwes J (2003) Parallel allocation of attention prior to the execution of saccade sequences. J Exp Psychol Hum Percept Perform 29: 882-896. [DOI] [PubMed] [Google Scholar]
  23. Gottlieb JP, Kusunoki M, Goldberg ME (1998) The representation of visual salience in monkey parietal cortex. Nature 391: 481-484. [DOI] [PubMed] [Google Scholar]
  24. Haenny PE, Schiller PH (1988) State dependent activity in monkey visual cortex. I. Single cell activity in V1 and V4 on visual tasks. Exp Brain Res 69: 225-244. [DOI] [PubMed] [Google Scholar]
  25. Hasegawa RP, Matsumoto M, Mikami A (2000) Search target selection in monkey prefrontal cortex. J Neurophysiol 84: 1692-1696. [DOI] [PubMed] [Google Scholar]
  26. Iba M, Sawaguchi T (2002) Neuronal activity representing visuospatial mnemonic processes associated with target selection in the monkey dorsolateral prefrontal cortex. Neurosci Res 43: 9-22. [DOI] [PubMed] [Google Scholar]
  27. Ito M, Gilbert CD (1999) Attention modulates contextual influences in the primary visual cortex of alert monkeys. Neuron 22: 593-604. [DOI] [PubMed] [Google Scholar]
  28. Judge SJ, Richmond BJ, Chu FC (1980) Implantation of magnetic search coils for measurement of eye position: an improved method. Vision Res 20: 535-538. [DOI] [PubMed] [Google Scholar]
  29. Kastner S, Nothdurft HC, Pigarev IN (1999) Neuronal responses to orientation and motion contrast in cat striate cortex. Vis Neurosci 16: 587-600. [DOI] [PubMed] [Google Scholar]
  30. Knierim JJ, van Essen DC (1992) Neuronal responses to static texture patterns in area V1 of the alert macaque monkey. J Neurophysiol 67: 961-980. [DOI] [PubMed] [Google Scholar]
  31. Kobatake E, Tanaka K (1994) Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex. J Neurophysiol 71: 856-867. [DOI] [PubMed] [Google Scholar]
  32. Li W, Thier P, Wehrhahn C (2000) Contextual influence on orientation discrimination of humans and responses of neurons in V1 of alert monkeys. J Neurophysiol 83: 941-954. [DOI] [PubMed] [Google Scholar]
  33. Luck SJ, Chelazzi L, Hillyard SA, Desimone R (1997) Neural mechanisms of spatial selective attention in areas V1, V2, and V4 of macaque visual cortex. J Neurophysiol 77: 24-42. [DOI] [PubMed] [Google Scholar]
  34. Martinez-Trujillo J, Treue S (2002) Attentional modulation strength in cortical area MT depends on stimulus contrast. Neuron 35: 365-370. [DOI] [PubMed] [Google Scholar]
  35. Maunsell JH, Sclar G, Nealey TA, DePriest DD (1991) Extraretinal representations in area V4 in the macaque monkey. Vis Neurosci 7: 561-573. [DOI] [PubMed] [Google Scholar]
  36. Mazer JA, Gallant JL (2003) Goal-related activity in V4 during free viewing visual search: evidence for a ventral stream visual salience map. Neuron 40: 1241-1250. [DOI] [PubMed] [Google Scholar]
  37. McAdams CJ, Maunsell JH (1999) Effects of attention on orientation-tuning functions of single neurons in macaque cortical area V4. J Neurosci 19: 431-441. [DOI] [PMC free article] [PubMed] [Google Scholar]
  38. McAdams CJ, Maunsell JH (2000) Attention to both space and feature modulates neuronal responses in macaque area V4. J Neurophysiol 83: 1751-1755. [DOI] [PubMed] [Google Scholar]
  39. McPeek RM, Keller EL (2002) Saccade target selection in the superior colliculus during a visual search task. J Neurophysiol 88: 2019-2034. [DOI] [PubMed] [Google Scholar]
  40. Mohler CW, Goldberg ME, Wurtz RH (1973) Visual receptive fields of frontal eye field neurons. Brain Res 61: 385-389. [DOI] [PubMed] [Google Scholar]
  41. Moore T, Armstrong KM (2003) Selective gating of visual signals by micro-stimulation of frontal cortex. Nature 421: 370-373. [DOI] [PubMed] [Google Scholar]
  42. Moran J, Desimone R (1985) Selective attention gates visual processing in the extrastriate cortex. Science 229: 782-784. [DOI] [PubMed] [Google Scholar]
  43. Motter BC (1993) Focal attention produces spatially selective processing in visual cortical areas V1, V2, and V4 in the presence of competing stimuli. J Neurophysiol 70: 909-919. [DOI] [PubMed] [Google Scholar]
  44. Motter BC (1994) Neural correlates of attentive selection for color or luminance in extrastriate area V4. J Neurosci 14: 2178-2189. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Murthy A, Thompson KG, Schall JD (2001) Dynamic dissociation of visual selection from saccade programming in frontal eye field. J Neurophysiol 86: 2634-2637. [DOI] [PubMed] [Google Scholar]
  46. Ogawa T, Komatsu H (2002) Context dependent neural processing in macaque V4 during multi-dimensional visual search task. Soc Neurosci Abstr 28: 559.4. [Google Scholar]
  47. Pasupathy A, Connor CE (1999) Responses to contour features in macaque area V4. J Neurophysiol 82: 2490-2502. [DOI] [PubMed] [Google Scholar]
  48. Pigarev IN, Rizzolatti G, Scandolara C (1979) Neurons responding to visual stimuli in the frontal lobe of macaque monkeys. Neurosci Lett 12: 207-212. [DOI] [PubMed] [Google Scholar]
  49. Reynolds JH, Desimone R (2003) Interacting roles of attention and visual salience in V4. Neuron 37: 853-863. [DOI] [PubMed] [Google Scholar]
  50. Reynolds JH, Chelazzi L, Desimone R (1999) Competitive mechanisms subserve attention in macaque areas V2 and V4. J Neurosci 19: 1736-1753. [DOI] [PMC free article] [PubMed] [Google Scholar]
  51. Reynolds JH, Pasternak T, Desimone R (2000) Attention increases sensitivity of V4 neurons. Neuron 26: 703-714. [DOI] [PubMed] [Google Scholar]
  52. Sato TR, Watanabe K, Thompson KG, Schall JD (2003) Effect of target-distractor similarity on FEF visual selection in the absence of the target. Exp Brain Res 151: 356-363. [DOI] [PubMed] [Google Scholar]
  53. Schall JD, Morel A, King DJ, Bullier J (1995a) Topography of visual cortex connections with frontal eye field in macaque: convergence and segregation of processing streams. J Neurosci 15: 4464-4487. [DOI] [PMC free article] [PubMed] [Google Scholar]
  54. Schall JD, Hanes DP, Thompson KG, King DJ (1995b) Saccade target selection in frontal eye field of macaque. I. Visual and premovement activation. J Neurosci 15: 6905-6918. [DOI] [PMC free article] [PubMed] [Google Scholar]
  55. Schein SJ, Desimone R (1990) Spectral properties of V4 neurons in the macaque. J Neurosci 10: 3369-3389. [DOI] [PMC free article] [PubMed] [Google Scholar]
  56. Schiller PH (1993) The effects of V4 and middle temporal (MT) area lesions on visual performance in the rhesus monkey. Vis Neurosci 10: 717-746. [DOI] [PubMed] [Google Scholar]
  57. Schiller PH, Lee K (1991) The role of the primate extrastriate area V4 in vision. Science 251: 1251-1253. [DOI] [PubMed] [Google Scholar]
  58. Schmolesky MT, Wang Y, Hanes DP, Thompson KG, Leutgeb S, Schall JD, Leventhal AG (1998) Signal timing across the macaque visual system. J Neurophysiol 79: 3272-3278. [DOI] [PubMed] [Google Scholar]
  59. Sereno AB, Maunsell JH (1998) Shape selectivity in primate lateral intraparietal cortex. Nature 395: 500-503. [DOI] [PubMed] [Google Scholar]
  60. Spitzer H, Desimone R, Moran J (1988) Increased attention enhances both behavioral and neuronal performance. Science 240: 338-340. [DOI] [PubMed] [Google Scholar]
  61. Tanaka K, Hikosaka K, Saito H, Yukie M, Fukada Y, Iwai E (1986) Analysis of local and wide-field movements in the superior temporal visual areas of the macaque monkey. J Neurosci 6: 134-144. [DOI] [PMC free article] [PubMed] [Google Scholar]
  62. Theeuwes J (1991) Cross-dimensional perceptual selectivity. Percept Psychophys 50: 184-193. [DOI] [PubMed] [Google Scholar]
  63. Theeuwes J, Godljn (2002) Irrelevant singletons capture attention: evidence from inhibition of return. Percept Psychophys 64: 764-770. [DOI] [PubMed] [Google Scholar]
  64. Theeuwes J, Kramer AF, Hahn S, Irwin DE, Zelinsky GJ (1999) Influence of attentional capture on oculomotor control. J Exp Psychol Hum Percept Perform 25: 1595-1608. [DOI] [PubMed] [Google Scholar]
  65. Thompson KG, Bichot NP, Schall JD (1997) Dissociation of visual discrimination from saccade programming in macaque frontal eye field. J Neurophysiol 77: 1046-1050. [DOI] [PubMed] [Google Scholar]
  66. Toth LJ, Assad JA (2002) Dynamic coding of behaviourally relevant stimuli in parietal cortex. Nature 415: 165-168. [DOI] [PubMed] [Google Scholar]
  67. Treisman AM, Gelade G (1980) A feature-integration theory of attention. Cognit Psychol 12: 97-136. [DOI] [PubMed] [Google Scholar]
  68. Treue S (2003) Visual attention: the where, what, how and why of saliency. Curr Opin Neurobiol 13: 428-432. [DOI] [PubMed] [Google Scholar]
  69. Treue S, Martinez Trujillo JC (1999) Feature-based attention influences motion processing gain in macaque visual cortex. Nature 399: 575-579. [DOI] [PubMed] [Google Scholar]
  70. Zeki S (1983) The distribution of wavelength and orientation selective cells in different areas of monkey visual cortex. Proc R Soc Lond B Biol Sci 217: 449-470. [DOI] [PubMed] [Google Scholar]

Articles from The Journal of Neuroscience are provided here courtesy of Society for Neuroscience

RESOURCES