Main

How does the brain link vision to action in real time? Experiments investigating this question have largely focused on reactive tasks, in which a stimulus triggers an immediate response1,2,3,4. Natural sensorimotor behavior, however, is not easily characterized as a cascade of stimulus-response associations5,6,7. For example, many actions occur with no immediate external event to trigger them. In these cases, one could always ask: why did the organism move at that precise moment and not a second earlier or later?

Of particular interest is the case where visual information is readily available and must be acted upon but does not include an abrupt cue for timing action. Consider a tennis player deciding when to initiate a swing: the looming ball is clearly visible, but there is no abrupt visual event that tells the player, “Swing now.” How does the brain determine when to move in such circumstances? Previous studies have addressed the initiation of action in the complete absence of external cues8,9,10 or when noisy sensory input must be integrated gradually over time11; however, a much more common yet largely unexplored situation is one in which non-noisy (suprathreshold) information is available but the initiation of action is not directly linked to any abrupt external event, as in the tennis example.

We trained monkeys in a task that directly compared visuomotor processes during proactive mental states, in which there is no abrupt external trigger for action, and reactive mental states. We found that when the visual world did not offer an abrupt external trigger for movement, a population of neurons in parietal cortex could provide the necessary internal trigger.

Results

Behavioral performance

Monkeys viewed repeated presentations of the same moving visual stimulus. What changed, across different blocks of trials, was the relationship between that stimulus and an associated arm movement. In proactive blocks, the monkeys pressed a lever before the moving stimulus reversed direction; in fact, their lever press caused the direction reversal. In reactive blocks, the monkeys pressed the lever after the direction reversal occurred, as a simple reaction to this abrupt cue. The task was designed so that the same visual and motor events occurred in all trials. The key difference, however, was that in proactive trials there was no abrupt external trigger for action, whereas in reactive trials there was an external trigger.

Each trial began with the monkeys fixating a small spot on the monitor. After 500 ms, two parallel bars (22° apart) appeared peripherally, with one bar placed inside or directly abutting the receptive field of the cell being recorded. A dot located between the bars appeared simultaneously and, after a brief delay, began moving at 13.5 deg s−1 in a straight path toward the bar inside the receptive field (Fig. 1a).

Figure 1: Behavioral task and neuronal recording sites.
figure 1

(a) Schematic of the visual stimulus. Cross, fixation point. Dotted oval, receptive field. (b) Three behavioral modes of the main task. (c) Schematic of macaque brain and recording sites. IPS, intraparietal sulcus; STS, superior temporal sulcus; MT, middle temporal area; MST, medial superior temporal area; LIP, lateral intraparietal area.

Full size image

The task alternated between three different blocks of trials: active, active delay and reaction (Fig. 1b). In the active block, when the monkeys depressed the lever in front of them, the dot reversed its direction of motion immediately. The monkeys' task was to turn the dot when it was within 3° of the bar, but not yet touching it; if the monkeys pressed the lever too early or too late, the trial was immediately aborted. The active-delay block was identical, except that the dot turned 200 ms after the lever press. In this block, the monkeys had to make their arm movements earlier in order to cause the dot to turn within the requisite spatial window. The active-delay block was also proactive, but it served to dissociate the time of the arm movement from the time of the motion reversal, so that neuronal activity could be more clearly related to one or the other event. In the reaction block the monkeys no longer had control over the motion reversals. Rather, they were required to press the lever immediately after noticing a motion reversal generated by the computer. In this block, we replayed dot trajectories from trials just completed in the active and active-delay blocks, so that the monkeys reacted to the identical visual events that they had proactively generated before (Methods).

In the active and active-delay blocks, the monkeys pressed the lever within a ∼230-ms window to perform the task successfully. In the reaction block, the monkeys almost never made premature movements as the dot approached the bar (<2% of trials). Instead, almost all lever presses occurred correctly, with a mean reaction time of 368 ms (s.d. = 48 ms) following motion reversals. These data support the distinction in how movements were initiated on proactive versus reactive trials (Methods).

The experiment was designed so that purely 'visual' cells and purely 'motor' cells would activate in all trials in a manner linked to the visual and motor events in the task, respectively. However, neuronal signals present during proactive, but not reactive, trials probably performed a more cognitive function related to generating or monitoring proactive movements, such as predicting the dot turn, calculating the time-to-contact between the dot and the bar, or internally triggering an action. By dissociating the timing of the lever press from the timing of external visual events, one can constrain the potential function(s) served by proactive neuronal signals. For example, a correlation of proactive activity with the time of lever press would suggest a role in the internal triggering of movement, whereas a correlation with the time of the dot turn would suggest an abstract visual-prediction function.

Single-neuron activity in parietal cortex

We recorded activity in four different cortical areas, the middle temporal area (MT), the medial superior temporal area (MST), area 5 and the lateral intraparietal area (LIP) (Figs. 1c and 2, and Methods). First, we show the response from a single MT neuron, aligned to the time of the dot's motion reversal (Fig. 3a). The stimulus was oriented such that the direction-selective neuron under investigation would be stimulated in the preferred direction as the dot approached the bar. The cell's firing was virtually the same in the three blocks, reflecting the consistent visual stimulation of the moving dot. Neurons in MT and MST tended to respond in a purely visual fashion in this task (Fig. 3b).

Figure 2: Structural T1-weighted MRI images and recording sites.
figure 2

1-mm sections are shown. Mineral oil–filled capillary tubes served as fiducial markers and appear as dark lines above the image of the brain. Colored dots, sites at which we encountered area 5 neurons (yellow), LIP neurons (red) and MT/MST neurons (blue) (Methods).

Full size image
Figure 3: Neuronal activity aligned to the time at which the dot turned.
figure 3

(a) Activity from a single MT neuron. Each tick mark in the rasters corresponds to a single spike time recorded in one correct trial. (b) Population-averaged histograms from MT/MST. (c) Activity from a single area 5 neuron. (d) Population-averaged histograms from area 5. (e) Activity from a single LIP neuron. (f) Population-averaged activity from LIP. For each LIP cell (n = 93), we measured the mean firing rate during a 250-ms window, starting 200 ms before the turn. We then calculated a modulation index as (R1 − R2)/(R1 + R2), where R1 denotes the mean firing rate in active and active-delay trials combined and R2 denotes the mean rate in reaction trials. Inset, distribution of the 93 indices (statistically significant values in black; P < 0.01). (g) Population-averaged histograms from the 40 LIP cells with significantly positive modulation indices. Vertical dotted line, lever-press time in the active-delay block. (h) Population-averaged histograms from the 53 LIP cells that did not have significantly positive modulation indices.

Full size image

Neurons recorded in area 5 had a completely different response profile, activating at the time of the arm movement. Area 5 neurons fired whenever the monkeys pressed the lever, in both proactive and reactive blocks (Fig. 3c,d).

An examination of a single neuron in area LIP shows that the cell fired a brief burst of spikes after the dot's turn in all blocks, which we interpret as a visual response to the motion reversal (Fig. 3e). In active and active-delay blocks, however, the neuron fired an additional, wider burst of spikes centered roughly on the time of lever press. This extra activity crudely resembled a motor response. However, unlike area 5 neurons, which fired whenever the arm moved, this LIP neuron did not respond to reactive arm movements (Fig. 3e, bottom).

Before the motion reversal, LIP cells tended to have higher activity in proactive blocks than in the reaction block (Fig. 3f). For each LIP neuron, we calculated a modulation index in a 250-ms window starting 200 ms before the turn (details in Fig. 3 legend). Positive values of the index indicated stronger activity in proactive blocks and negative values indicated stronger activity in the reaction block. The distribution of 93 indices (one value per cell) was significantly shifted away from zero, in the positive direction (t-test, P = 6.3 × 10−6, inset in Fig. 3f). Among individual neurons, 40 were significantly more active in proactive blocks than in the reaction block, whereas only 13 showed the opposite trend (t-test, P < 0.01).

What is the function of increased activity in proactive conditions? To gain insight, we examined the temporal properties of the modulation in the 40 LIP cells with significantly positive modulation indices (Fig. 3g). Like the single-neuron example (Fig. 3e), the population response of these cells included a clear surge of activity centered roughly on the time of the lever press, but this surge was absent in the reaction block (Fig. 3g, thick arrow). Notably, in active and active-delay blocks, the rising response achieved a consistent spike rate at the time of the lever press (Fig. 3g, thin arrows), suggesting that activity in these neurons might reach a uniform spike rate threshold to trigger a proactive movement. The population response of the remaining 53 LIP cells (those lacking significantly elevated activity) did not exhibit this tendency: these cells fired more consistently across the three blocks (Fig. 3h).

A parietal signal for the triggering of action

If a subset of LIP neurons indeed function to trigger action, their activity should predict the precise time of proactive hand movements whenever these movements occur during a trial. Again analyzing the 40 LIP cells with elevated modulation indices, we plotted the population-averaged activity from proactive trials, aligned to the start of dot motion and grouped by the time of lever press (Fig. 4a). These data included premature and late lever presses and therefore spanned more than 800 ms of arm-movement initiation times. The activity of these 40 LIP neurons evolved quite differently depending on the ultimate time of lever press, yet always reached a consistent firing rate of ∼40 spikes per s at the time of the hand movement. Note that these cells were not preselected for having this effect.

Figure 4: Neuronal activity aligned to the start of dot motion and grouped by time of arm movement.
figure 4

(a–d) The times of lever presses from all active and active-delay trials were pooled into one distribution (light gray histogram) and divided into 15 groups, with equal numbers of trials per group. Trials in which the monkeys pressed the lever after the dot hit the bar were treated separately (dark gray bins). The mean lever-press time from each group is shown as a vertical colored line. Spike trains from individual trials were convolved with a Gaussian kernel (s.d. = 20 ms) and averaged together with all other trials in a group. Data before the break in the curves were aligned to the start of dot motion. Data after the break were aligned to the lever press for averaging, and then each curve was shifted in time to terminate at the appropriate mean lever-press time, referenced to the start of dot motion, for that group. This essentially realigns the data to the start of dot motion but allows us to better estimate the ends of the curves. On any given trial the dot could appear at one of two locations before moving. Only data from a single start location are shown, but results were very similar for the other half of trials. (e–h) Single-cell data are displayed as in a–d (Methods). The time points at which curves crossed the horizontal threshold line were regressed against the associated mean lever-press time for each group (insets; Methods). (i–k) Distributions of single-cell regression slopes. Only cells with well-estimated slopes (s.e.m. < 0.25) were included (64 of 93 LIP cells, 42 of 67 MT/MST cells and 88 of 94 area 5 cells).

Full size image

Whereas the responses of many neurons in our study could easily be related to classic sensory or motor processing, the activity of these 40 LIP neurons could not. For example, the 53 LIP cells that lacked elevated pre-turn activity in proactive blocks fired similarly regardless of lever-press time (Fig. 4b). MT/MST responses were likewise uncorrelated with the time of lever press and presumably reflected the visual stimulation of the moving dot, which was identical on all trials (Fig. 4c). Area 5 activity was correlated with the time of lever press (Fig. 4d), but rose at the same rate regardless of when the arm movement occurred during a trial. In contrast, the firing rate buildup of the 40 LIP neurons with elevated activity on proactive trials ascended at different rates: when activity rose quickly the monkeys moved early, and when activity rose more slowly the monkeys moved late (Fig. 4a). This profile describes a cognitive process that could evolve toward a threshold to trigger action, rather than a stereotyped motor response2,4,11. We refer to this process as 'proactive timing'.

Threshold analysis in single cells

The population-averaged activity suggested a functional division in LIP, with only a subset of cells involved in proactive timing. To test this issue directly, we plotted the proactive responses of single neurons grouped by the time of lever press (Fig. 4e–h). For each cell we then extracted the time points at which neuronal activity crossed a threshold line drawn through the data, and regressed the threshold-crossing times against the corresponding arm-movement times (Methods). For cells that responded the same regardless of lever-press time, the regression slope would be 0; for cells whose activity crossed an identical threshold level at the time of an arm movement, the regression slope would be 1. Regression slopes in LIP were distributed with a distinct cluster around 1 and a second cluster below 1, supporting the notion that a subset of LIP neurons participates in proactive timing and a distinct subset is not involved (Fig. 4i). As expected, regression slopes from MT/MST and area 5 were unimodally clustered around 0 and 1, respectively (Fig. 4j,k).

LIP cells with regression slopes near 1 also tended to have higher activity in the proactive blocks than in the reaction block, as expected (correlation analysis between the modulation index calculated in Fig. 3f and the regression slope, r = 0.56, F62,62 = 3.51, P = 1.8 × 10−6; Fig. 5a). Furthermore, when restricting the regression slope distribution to only those LIP cells particularly well fit by the regression model (s.e.m. < 0.15), the bimodality in the distribution was even more conspicuous (Fig. 5b). That is, some cells had regression slopes of 0.7 or below and other cells had slopes of 0.8 or above, with fewer values near the boundary. How distinct were these two pools of neurons? We set a boundary by eye to a slope value of 0.75. The five cells with regression slopes immediately on either side of this border exhibited the bimodality that was evident across the entire population, where some cells were highly predictive of lever-press time (Fig. 5c) and others responded consistently regardless of lever-press time (Fig. 5d). These results support a genuine functional division within the LIP population.

Figure 5: Functional bimodality in LIP.
figure 5

(a) The histogram is identical to that in Figure 4i, except that data from cells with significantly higher activity (P < 0.01) in proactive blocks than the reaction block are shown in gray. (b) Same as in a, but including only neurons with s.e.m. < 0.15. (c,d) Population-averaged activity grouped by time of lever press for the five cells with regression slopes (c) closest to the boundary but above it or (d) closest to the boundary but below it.

Full size image

Most area 5 neurons and even a few MT/MST neurons had regression slopes near 1 (Fig. 4j,k). However, the temporal characteristics of activity in these cells were inconsistent with proactive timing. There are various time courses by which neuronal activity could rise to reach a consistent firing rate at the moment of action. For example, activity might rise at the same rate but with different onset latencies, a result expected for a stereotyped motor response (Fig. 6a, top). Alternatively, activity might rise at varying rates (Fig. 6a, middle). These two profiles can be dissociated by examining regression slopes at thresholds well below the highest possible threshold for each cell. At lower thresholds, area 5 neurons still had regression slopes near 1, consistent with these cells showing a stereotyped motor response (Fig. 6b, top). LIP neurons with proactive timing activity, however, had regression slopes intermediate between 0 and 1 at lower thresholds, consistent with their activity rising at variable rates (Fig. 6b, middle). Notably, the four MT/MST neurons with regression slopes near 1 at high thresholds had regression slopes near 0 at lower thresholds (Fig. 6b, bottom). For these few neurons, activity may have risen similarly on all trials but then splayed out at the last instant, perhaps due to a brief modulation around the time of lever press (Fig. 6a, bottom). These neurons would presumably not provide signals early enough for triggering the arm movement.

Figure 6: Single-neuron activity in LIP is consistent with a variable rate of rise to threshold.
figure 6

(a) Three hypothetical patterns of neuronal activity that would yield a consistent threshold firing rate at the time of lever press (that is, regression slopes of 1 in Fig. 4i–k). To differentiate among the three hypothetical response profiles, we repeated the threshold analysis described in Figure 4 for lower thresholds (Methods). (b) In area 5, regression slopes were unchanged between the high and low thresholds, consistent with activity rising in parallel. In LIP, regression slopes were shallower at the lower threshold and generally intermediate between 0 and 1, consistent with proactive timing activity. In MT/MST, the four neurons with regression slopes near 1 at high thresholds had slopes close to 0 at lower thresholds, consistent with visual responses that were modulated only briefly at the time of the arm movement.

Full size image

Relative timing of LIP signals and area 5 signals

One way in which proactive timing activity could trigger a movement is by reaching a consistent threshold firing rate. If so, it is essential that the threshold be reached before the movement is actually initiated. The average motor-like burst in area 5 began ∼300 ms before the lever press (Fig. 7a, arrow). This time point might serve as a good surrogate for the initiation of the motor process. Could LIP activity be described as reaching threshold at or before this time? Examining the proactive timing data (Fig. 4a), we terminated the 15 population-averaged curves 300 ms before their associated mean lever-press times, and found that the activity was at a uniform level at this time point (Fig. 7b). To quantify this trend, we performed the same analysis in individual neurons—terminating the 6 curves from each cell 300 ms before the lever press—and found that only 2 of 26 neurons had a linear least-squares fit to the terminal firing rates with a slope that differed significantly from zero (P < 0.01, analyzing neurons with regression slopes above 0.8 in Fig. 4i). Examining the data 400 ms before lever press, still only 2 of 26 neurons had fits that differed significantly from zero (P < 0.01). In contrast, activity of the more visual-like LIP cells and MT/MST cells did not show this uniformity (Fig. 7c,d).

Figure 7: Neuronal activity 300 ms before the lever press.
figure 7

(a) The population-averaged activity from area 5 neurons is displayed aligned to the time of lever press. Active and active-delay trials have been combined. The major firing rate upswing began ∼300 ms before the lever press (arrow). (b) LIP data from Figure 4a are shown, but each of the 15 curves is terminated 300 ms before its associated mean lever-press time, and the firing rate at that instant is indicated with a black dot. The firing rate reached a relatively consistent level 300 ms before the lever press, irrespective of when the arm movement occurred during a trial. (c,d) LIP data from Figure 4b and MT/MST activity are shown in the same manner as in b. Activity in these neurons did not achieve a consistent threshold firing rate 300 ms before the lever press.

Full size image

Furthermore, irrespective of the rise-to-threshold model, one can ask when in a trial does LIP activity begin to predict the timing of an upcoming lever press. We regressed the firing rates of the 15 proactive timing curves (Fig. 4a) against their associated lever-press times in 1-ms bins beginning at the start of dot motion. The regression was statistically significant 256 ms after the dot began moving and remained significant until the end of the trial (P < 0.01). In contrast, the earliest upswing in the area 5 data (Fig. 4d) began ∼600 ms after the dot began moving. Thus, regardless of the specifics of the threshold crossing formalism, signals in LIP were present early enough to serve a role in the timing of action.

Discussion

LIP cells have been shown to ramp up their firing rates in expectation of visual events, such as the onset or dimming of a stimulus in the receptive field12,13 or the onset of visual motion14. Thus one possibility is that ramping activity in our task anticipated the turn of the dot, which was a highly salient visual event. This interpretation, however, provides a poor explanation for our active and active-delay data. In these two modes, we dissociated the time of the arm movement from the time of the dot turn and found that the proactive ramp-up predicted the time of the motor event and not the visual event. That is, for active-delay trials, the response began to fall after the time of the arm movement; it did not continue to rise up until the later time of the dot turn (Fig. 3g). In previous studies, external visual events were usually linked closely in time to corresponding movements of the eyes or limbs. Our findings suggest that the anticipatory activity in these studies could have reflected the beginnings of a proactive triggering process by the monkeys, rather than exclusively reflecting the visual stimulus. Notably, however, proactive ramp-ups in the previous tasks would necessarily have been cut short by the external cues for movement, which were invariably provided to the monkeys. These cues could have obscured the tight coupling between neuronal activity and movement time, as characterized here.

Another possibility is that the proactive timing signals anticipated a different visual event, namely, when the dot reached a certain distance from the bar. Given that the dot was moving at a constant speed, this would be akin to signaling 'time-to-contact', a visual variable that has been hypothesized to serve a role in the proactive timing of action15. This interpretation also seems unlikely. The moving dot was much farther away from the bar for early presses than for late presses (∼11°), yet LIP activity achieved the same firing rate at the time of the arm movement (Fig. 4a). Thus the proactive timing signal provided an extremely inaccurate estimate of the dot's position or, equivalently, of time-to-contact. One could, of course, argue that the monkeys solved the task by using this inaccurate estimate of the dot's location. However, the firing rate of neurons in MT/MST (Fig. 4c) and the more visual-like LIP neurons (Fig. 4b) provided a much more reliable estimate of the dot's position as a function of time. It is doubtful the monkeys ignored these signals and used a much poorer estimator of the visual stimulus. A more parsimonious view is that proactive timing reflects an internal process linked to the time of the arm movement.

Past experiments have shown that a minority of LIP cells have activity associated with arm-reaching movements or with both reaches and saccades16. Although proactive timing could be interpreted as a growing intention to move the arm, our results go considerably further. The LIP signals showed extraordinary temporal specificity: neurons reached a consistent firing rate before and at the moment of action, despite more than 800 ms of movement-time variability. Moreover, arm movements initiated early in a trial were associated with firing rates that rose quickly, whereas later arm movements were associated with firing rates that rose more gradually. These temporal properties suggest a role for LIP in the real-time triggering of actions, above and beyond reflecting the intention to make those actions.

It has also been argued that LIP activity is not linked to particular movements but rather functions to represent the salience of a specific portion of retinotopic space17. The proactive timing process we observed in LIP is probably distinct from encoding the salience of (or intention to look toward) specific parts of space. First, the moving dot should be salient for guiding behavior in both proactive and reactive blocks, yet we did not observe a buildup of activity before reactive lever presses. Second, whereas our LIP population showed spatially selective delay activity in a memory saccade task—a physiological signature of LIP and a measure of spatial salience—the strength of the delay period modulation was uncorrelated with the strength of the proactive timing effect in individual neurons (Fig. 8). Another potential measure of salience in the memory saccade task is the strength of a cell's visual on-response, but this too was uncorrelated with proactive timing (Fig. 8, legend). Third, the division of labor we observed in LIP, with some cells participating in proactive timing and others not, is inconsistent with a single functional interpretation. If anything, the more visual-like LIP cells (Fig. 4b) seemed to be reflecting spatial salience. For example, one could easily determine the location of the most obvious visual target, the moving dot, from their population response.

Figure 8: Proactive timing activity does not correlate with delay period modulation in a memory saccade task.
figure 8

In the memory saccade task, the monkeys made saccades to the remembered locations of small targets flashed in the periphery (Methods). For 49 LIP cells that had a well-estimated regression slope (s.e.m. < 0.25), we also tested the response to a memory saccade target placed within the response field. The middle panel shows the population-averaged responses to this target (black curve) and to another target placed equidistantly in the opposite hemifield (gray curve). For each cell, we measured the mean firing rate during the final 700 ms of the delay period, for both saccade targets (the total delay interval was a fixed 1,050 ms). We then calculated a modulation index as (R1 − R2)/(R1 + R2), where R1 and R2 denote the mean firing rate for the target flashed in and away from the response field, respectively. This delay period modulation index is plotted against the cell's regression slope from the main task (bottom). There was no obvious relationship between these two measures (r = −0.12, F47,47 = 1.27, P = 0.42). We also calculated a second modulation index capturing the strength of each cell's visual on-response, with R1 as the mean firing rate in a 300-ms window immediately following the onset of the memory saccade target in the response field, and R2 as the baseline firing rate in a 200-ms window immediately preceding target onset. This measure was likewise uncorrelated with the cells' regression slopes from the main task (r = 0.01, F47,47 = 1.03, P = 0.93, data not shown).

Full size image

What is the mechanism for the proactive buildup? Previous work has described a similar rise in LIP activity before eye movements made to report the perceived direction of motion in a noisy moving stimulus11. The authors suggested that ramping activity in LIP reflects the mathematical integration, over time, of firing rate differences between pools of neurons in area MT (ref. 18). In our task, however, the motion stimulus was not noisy, and as a consequence MT/MST neurons fired in a consistent manner in all trials. Therefore, the marked variability we observed in the rate of rise of LIP activity is unlikely to have resulted from an integration of sensory responses. It is more likely that proactive timing signals are not linked to the specifics of the visual input but rather serve a more general role in timing movements.

At first glance it seems counterintuitive that a retinotopic area, such as LIP, contains signals immediately useful for timing arm movements. For example, if the monkey shifted its gaze before initiating the arm movement, it is likely that the rise-to-threshold would be disrupted. However, many real-world sensorimotor behaviors include explicit periods of relatively stable and reproducible fixations19,20. For example, we generally reach for targets that we are stably fixating21,22. Retinal coordinate signals in LIP could therefore be used by the skeletomotor system in these circumstances23,24. Although LIP has conspicuous outputs to brain areas involved in saccadic eye movements, there are also weaker projections to hand movement centers, such as area 5, as well as indirect routes to the skeletomotor system via projections to the ventral intraparietal area and anterior intraparietal area25,26,27. Alternatively, the proactive timing activity we observed in LIP might reflect a more general process that is unaffected by changes in gaze. Abstract timing signals and anticipatory ramping have been observed in a variety of cortical areas, as well as in the cerebellum and basal ganglia28,29,30,31. It will be important to examine whether anticipatory signals in these brain areas also show a precise relationship to the proactive timing of movements, and, if so, whether they do so independently of where the animal is looking. Regardless, our data demonstrate that when the external world does not provide a trigger for action, an internal mental process—proactive timing—will.

Methods

Physiological techniques.

Two male rhesus monkeys (Macaca mulatta, ∼10 kg) were surgically implanted with a head post, scleral search coil and recording chamber. Animal care and all surgical and experimental procedures followed Harvard Medical School and US National Institute of Health guidelines. The chambers were dorsally positioned at stereotactic coordinates P3, L10 in the hemisphere contralateral to the hand used for pressing the lever. Electrophysiological recordings were made from single neurons using tungsten microelectrodes (FHC, 75 μm shaft diameter) and a guide-tube/grid system. Spike times were recorded with 1 ms resolution. Eye positions were sampled at 200 Hz.

Structural magnetic resonance imaging (MRI) facilitated the localization of recording sites. Capillary tubes filled with mineral oil were placed inside the recording grid during imaging and served as fiducial markers to determine the anterior-posterior and medial-lateral location of each electrode penetration. In the reconstruction of recording sites (Fig. 2), each colored dot reflects one recording site, although we often conducted multiple penetrations targeting each site. The depth of each dot was constrained to overlap gray matter and to honor the gyrus/sulcus transitions and gray matter/white matter transitions that we encountered during that recording session. For example, in a penetration at stereotactic coordinate P0,L7, we encountered motor-like activity, followed by a quiet period consistent with a sulcus, then visual/cognitive activity, and then white matter. Along that trajectory, we thus placed one (yellow, area 5) dot in the medial bank of the intraparietal sulcus and one (red, LIP) dot on the lateral bank.

Physiological criteria had a more important role in assigning individual neurons to particular cortical areas. In penetrations targeting the superior temporal sulcus, neurons in areas MT and MST were distinguished by highly direction-selective responses to spots and bars used for hand mapping of the receptive fields. In penetrations targeting the intraparietal sulcus, neurons recorded in the medial bank responded predominantly to hand movements and were assigned to area 5, whereas neurons in the lateral bank were more visually responsive and were assigned to area LIP (quantified below). LIP neurons were also identified by their showing sustained activity during the delay period of a memory saccade task (see below) or by being in immediate proximity to neurons with sustained delay activity. Area 5 neurons were generally completely unresponsive in the memory saccade task. In one monkey, we also encountered cells in the fundus of the intraparietal sulcus with highly direction-selective visual responses. These cells were most likely in the ventral intraparietal area, and are not discussed further. Of 94 cells assigned to area 5, 6 were more likely in surface cortex, just outside the intraparietal sulcus. Area 5 neurons did not show passive visual responses, and we could not map receptive fields. The stimulus was therefore positioned at a random location and orientation in the contralateral hemifield for these cells. It is likely that our area 5 recordings overlapped areas 5D/5V (ref. 32) and PE/PEip (ref. 33). They were more anterior than the parietal reach region24,34.

After isolating a neuron, we collected data in several secondary tasks to characterize the basic properties of the neuron. In the memory saccade task, targets appeared in one of eight possible locations, evenly distributed at 45° intervals, at 10° eccentricity. Neurons in LIP showed responses in this protocol that were typical of those reported previously35,36. In particular, we found that about half of the LIP neurons had elevated activity during the memory delay period. Our main results held when we separately analyzed LIP neurons that had significantly elevated delay activity in the memory saccade task and those that did not. The memory saccade task also provided a clear dissociation between LIP and area 5. For example, an analysis of variance (ANOVA) showed that 60 of 91 LIP neurons, but only 1 of 93 area 5 neurons, had visual responses that were significantly modulated by the location of the flashed target in the memory saccade task (P < 0.01, comparing firing rates across the eight locations in a 300-ms window following target onset). The percentage of visually modulated LIP cells in the memory saccade task (66%) might have been higher had we optimized the eccentricity of the flashed targets for each neuron; nevertheless, this percentage is comparable with past studies35,36. For cells in MT, MST and LIP, we typically mapped receptive fields quantitatively using a sparse-noise stimulus spanning 20° × 20° of visual angle. An ANOVA showed that 86% of LIP neurons and 98% of MT/MST neurons tested in this manner were significantly modulated by the location of the flashed stimulus (P < 0.01, comparing firing rates across the 25 stimulus locations in a 300-ms window following stimulus onset). As a final test, we also conducted preliminary trials of the main task in which we presented the bars at various orientations and locations in the reaction mode; this helped us position the main task stimulus in the most responsive retinotopic location, especially for the few cells that did not have clear receptive fields by other measures.

Main behavioral task.

The monkeys always performed the blocks in the following order: active, active delay, reaction. We only analyzed neurons for which the monkeys completed each block at least twice. On half the reaction trials (pseudorandomly interleaved), the monkeys reacted to dot turns that occurred at random, exponentially distributed time points in the dot's trajectory; in these cases the dot could turn well before or well after the normal turn zone. These trials were included to ensure that the monkeys could not proactively time their arm movements in the reaction block, and are not discussed further. On reaction trials, the allowed reaction-time window was 135–500 ms after the motion reversal (expanded to 135–550 ms for some sessions). In all blocks, the monkeys had to maintain gaze within 0.95° of the fixation point during the entire time course of a trial or that trial would abort without reward.

In any given session, the monkeys performed the task by either pressing a lever or releasing a touch bar. For simplicity, in the main text we refer to all arm movements as lever presses. Each trial actually consisted of two dot turns, one near each visible bar. Neurons in areas MT, MST and LIP did not generally respond to the second motion reversal as the dot was outside the receptive field, and area 5 neurons responded similarly to the first and second arm movements. We only consider data from the first dot turn in this report.

In the two proactive blocks, active and active delay, the monkeys successfully performed the first dot turn on 50.0% of trials, with other presses coming either too early (23.3%) or too late (26.7%). In the reaction block, the monkeys almost never made proactive movements as the dot approached the bar (1.5% of trials); rather, 95.6% of lever presses occurred correctly, within a brief reaction-time window following dot turns (2.9% of reactions were late). These behavioral percentages exclude trials in which the monkeys broke fixation (12% of trials). The reaction-block percentages also exclude trials that preceded the first correctly executed trial in a block. Before this trial, the monkeys invariably attempted to proactively turn the dot, not realizing that the block had changed.

The difference in performance between proactive and reactive blocks could have produced a difference in reward expectation. However, our main results focus on a process evident within proactive blocks specifically, so reward probability should not have been an issue. Furthermore, we describe a dynamic process that peaked at the time of the first arm movement within each trial, whereas the primary reward (juice) was delivered following the second arm movement, which came >1 s later.

We performed a uniform post-hoc screen on all recorded cells, selecting for analysis those neurons that activated as the dot approached the bar in the active block (for example, direction-selective neurons for which the initial trajectory was in the null direction were excluded). To perform the screen, we defined a pair of 250-ms-wide windows. The first window started 700 ms before the dot turn. The second window started 200 ms before the dot turn. In this report. we only consider neurons that fired significantly more spikes in the second window as compared to the first (paired t-test, P < 0.01).

Statistical analysis.

We used standard methods of hypothesis testing and linear regression. For the threshold procedure (Fig. 4e–k), we analyzed the data from each individual neuron as follows. We divided the active and active-delay trials into six groups based on the time of lever press, with equal numbers of trials per group (we excluded trials in which the monkey pressed the lever after the dot hit the bar). Spike trains from each trial were convolved with a Gaussian kernel (s.d. = 40 ms) and averaged together with other trials in their group, generating six averaged-response curves. A horizontal threshold line was drawn through the data, extending from 200 ms before the earliest mean lever-press time to 200 ms after the latest mean lever-press time. The line was positioned at the highest firing rate at which at least four curves still crossed the line with positive slopes (0.5 spikes per s resolution). The time points at which curves crossed this threshold were regressed against their associated mean lever-press times. To reduce noise, we repeated the procedure for two other thresholds, 0.5 spikes per s and 1 spike per s lower than the original threshold, and the three slope parameters and their standard errors were averaged together to generate a single slope and standard error value for each cell. The lowest of these three thresholds and its associated regression are shown (insets in Fig. 4e–h).

The threshold-regression procedure revealed at least some cells in all cortical areas with regression slopes near 1. To see if these responses could be distinguished among different cortical areas, we performed the same analysis at variable thresholds (Fig. 6). We raised a horizontal threshold line from 0 spikes per s to the maximum spike rate exhibited by each cell, in increments of 0.5 spikes per s. At each level, we inspected whether at least four of the six curves crossed the threshold line with positive slopes. If so, we extracted the threshold-crossing times of these four (or more) curves and regressed these values to the associated mean lever-press times. To differentiate between the three hypothesized response profiles (Fig. 6a), we averaged the regression slopes (and their standard errors) at three low thresholds, two-thirds of the way down to the lowest possible one. We selected cells for comparison if they had regression slopes above 0.8 in the original, high-threshold analysis, and additionally, if their regression slopes were well estimated at both high and low thresholds (s.e.m. < 0.25).