We often take for granted the ease with which we move our bodies. Yet, how our motor system performs even a simple task such as picking up a coffee mug remains a challenging problem scientifically. We move with considerable trial-to-trial variability, yet we successfully perform such tasks with speed and grace. In contrast, robots possess greater precision and consistency in their motions, but are nothing short of clumsy and awkward when picking up objects. Why are body movements that are so variable consistently successful? In this issue, Todorov and Jordan1 provide a new theory for motor coordination based on optimal feedback control that may be a major step forward in developing a single, cohesive framework for interpreting motor function.

One important feature captured by this theory is that motor commands are corrupted by noise, and that this signal-dependent noise increases with signal size2,3. Harris and Wolpert4 recently demonstrated the importance of considering noise in control of eye and limb movements. They were able to predict the bell-shaped velocity profiles and relatively straight hand trajectories that are observed experimentally5,6 by using a model that minimizes noise.

A second key feature in the Todorov and Jordan1 theory is the idea that the motor system can be modeled based on the principles of optimal feedback control (Fig. 1). The most important feature of this approach is that optimization techniques are used to find the feedback control law that minimizes errors in task performance. This control law is specific for each motor task, so that the CNS must select the appropriate control law for each task. If the goal is to maintain the hand at one location in space, feedback signals on the state of the system (joint position, velocity and force) for motor corrections are optimized specifically to maintain a constant hand position, and these control laws reflect the physical properties of the motor periphery. The authors capture this feature of optimal feedback control by using what they call the minimum intervention principle, which postulates that deviations from an average hand trajectory (or position) are only corrected if they interfere with task performance. By correcting only task-relevant errors, the model minimizes the potential effects of noise.

Figure 1: Diagram for implementing optimal feedback control as proposed by Todorov and Jordan1.
figure 1

Rebecca Henretta

The optimal feedback control law is selected by the CNS based on the specific task. An optimal estimate of the state of the system (positions, velocities and forces) is based on sensory feedback (which is delayed and noisy), efference copy of prior controls signals and forward internal models of the limb12. Noise is introduced to both motor and sensory signals.

Full size image

Todorov and Jordan1 illustrate the notion of optimal feedback control with a very simple example, a task whose goal is that the sum of two control signals equals two. The nominal strategy to minimize signal size is to set each signal to one. However, each of these signals can be corrupted by noise. A crucial question is how should the control law respond to such errors? The optimal strategy is that its adjustments should depend on fluctuations in both signals. If both control signals equal 1.1 (assuming no noise in the sensory signals), then the optimal strategy is that both control signals should be reduced toward 1. In contrast, if one control signal is 1.1 and the other is 0.9, then the optimal strategy is to not intervene because the goal of the task, that their sum equals 2, has been attained. The byproduct of the optimal control scheme is that the variability of the individual control signals becomes greater than the variability of their sum.

Reducing task variability at the expense of variability elsewhere in the system is also a key feature of human and animal motor coordination. For example, there are many different arm configurations that a given subject can use to maintain a steady aim at a target with a hand-held laser pistol. In such tasks, variability among these task-invariant arm configurations over time is very large compared to variability in joint configurations that interfere with pointing the laser7. That is, variability is tolerated as long as it does not interfere with task performance. The key proposal of Todorov and Jordan1 is that this differential management of variability during motor behavior occurs because it is the optimal solution for the task.

If the motor system puts such a premium on managing the position of the hand over the position of the joints during postural tasks like pistol shooting, it seems reasonable to believe that in a task such as reaching, the motor system will attempt to control hand trajectory. Although many hypotheses assume that the trajectory is explicitly controlled5,8, such models fail to capture another important feature of reaching movements, the considerable trial-to-trial variability in hand trajectory9. Todorov and Jordan1 capture this hand path variability for a throwing task using a model based on optimal feedback control. Such variability in hand trajectory is tolerated because it does not interfere with task performance, but it is inconsistent with explicit trajectory planning. Optimal feedback control does not plan the hand trajectory, which instead simply emerges from the optimal control law for the task. What has often been interpreted as a sign of sloppy control by the brain may actually reflect the optimal strategy for controlling body movements.

In effect, Todorov and Jordan argue that the feedback control law is not fixed, but is malleable and can be set based on the motor task. If this is true, a major question becomes how the motor system can learn these optimal control laws for myriad motor behaviors performed by an individual.

The new article1 provides a cohesive framework for interpreting motor coordination and provides interesting examples of how optimal feedback control can explain many observations on coordinated movement. However, use of stochastic optimal feedback control as a model of motor control comes with a large computational price, requiring challenging mathematical contortions to solve even the simplest of linear control problems. As a result, the musculoskeletal system in some cases must be modeled as point masses providing only motion along a single direction. It seems a bit ironic that a theory illustrating the importance of considering the properties of the musculoskeletal system for motor control must use incredibly simplistic models of the motor periphery! This should not be seen as a downside of the theory proposed by Todorov and Jordan1. Rather, this limitation simply reflects the lack of existing mathematical tools to apply optimal feedback control to complex non-linear systems, like our motor system. However, the intuitive value of the many examples presented in this paper cannot be ignored.

Although it may be comforting to assume that emergent patterns of motor behavior reflect the optimal strategy for a given task, that conclusion may not apply to all cases. The neural circuits to control movement are very distributed and complex, and they presumably are based in part on evolutionary baggage. The Todorov and Jordan optimal control theory tends to ignore this inherent hierarchical organization10. It seems reasonable to believe that motor circuitry itself can influence strategies for a given task, perhaps because the motor circuitry cannot be entirely optimized for each individual task. Instead, certain features of the circuit may be optimal only when the complete motor repertoire of humans is considered, much like the conclusion that the distribution of muscle spindles may be optimal only by considering the complete behavioral repertoire of the animal11. However, optimizing for such global cost functions is likely to be quite a challenge.