Control of Hand Shape — Neural Control of Movement Laboratory

Control of hand shape and multi-digit forces

In order to understand how the hand’s multiple degrees of freedom are controlled, we performed a series of behavioral studies on the control of multi-digit joints for hand shaping and multi-digit forces. The work on whole-hand shape and five-digit grasp forces was based on earlier work by the NCML PI at the University of Minnesota revealing covariation patterns in finger joint angles and digit forces^1,2,3 (Fig. 1). This work inspired grasp planning algorithms⁴ and the design of robotic and prosthetic hands;⁵ (for review see ⁶). At Arizona State University we performed follow-up studies where we found that continuous visual feedback of the hand is not necessary to modulate hand shape to object geometry.⁷ Further investigations revealed that, although digit force synchrony patterns occur regardless of handedness and object center of mass predictability,⁸ they are task dependent. Specifically, they do not occur when there is no need to generate a stable grasp and minimize the risk of object slip⁹ (Fig. 2).

Figure 1. Left: Covariation of finger joint angle pairs recorded across 57 hand shapes used to grasp imagined objects. Right: First two principal components of hand postures. From: Santello, Flanders and Soechting (1998). — **Figure 1**. Left: Covariation of finger joint angle pairs recorded across 57 hand shapes used to grasp imagined objects. Right: First two principal components of hand postures. From: Santello, Flanders and Soechting (1998).

Figure 2. Left: Grip device used to measure multi-digit normal and tangential (load) forces. Right: Distribution of phase angles as a function of frequency of normal forces exerted by all digit pairs. From: Santello and Soechting (2000). — **Figure 2.** Left: Grip device used to measure multi-digit normal and tangential (load) forces. Right: Distribution of phase angles as a function of frequency of normal forces exerted by all digit pairs. From: Santello and Soechting (2000).

How and where we grasp object depends on the interaction among several factors, including object shape, size, and intended use. In a series of studies examining hand kinematics, we found that fingertip positions in a whole-hand grasp are sensitive to the object mass distribution and its predictability¹⁰ (Fig. 3). Interestingly, when participants are cued about the object center of mass, they can modulate fingertip position accordingly, but not the forces.¹¹ We also found that the ability to modulate digit position to task requirements (lifting the object straight by countering an external torque) is affected by Parkinson’s disease.¹²

Figure 3. Left: Multi-digit contact points distribution as a function of center of mass and its predictability. Right: time course of object roll (A) and peak object roll (B) as a function of center of mass and its predictability. From: Lukos, Ansui… — **Figure 3.** Left: Multi-digit contact points distribution as a function of center of mass and its predictability. Right: time course of object roll (A) and peak object roll (B) as a function of center of mass and its predictability. From: Lukos, Ansuini and Santello (2007).

To further understand the coordination between fingertip position and forces, we investigated the coordination of digit forces as a function of digit placement in two-digit grasping and manipulation. To address this question, we used a novel custom-built sensorized devices (Fig. 4) to obtain a complete biomechanical description of forces and torques in three dimensions, which allowed us to compute the center of pressure of the thumb and index finger. We found that subjects (a) control digit placement in an anticipatory fashion to counter an external torque caused by an object’s asymmetrical mass distribution,¹³ (b) concurrently modulate digit force and position on a trial-to-trial basis (Fig. 5), (c) can transfer learned digit force-position mapping to a novel grip type,¹⁴ and (d) visual feedback is not necessary to coordinate digit forces to position.¹⁵ These findings are important as they emphasize the involvement of different sensorimotor mechanisms when subjects can choose (unconstrained) digit placement (see ^{16, 17} for follow-up studies and analyses). The phenomenon of digit force-to-position modulation has recently been extended from two- to five-digit grasping.¹⁸ This work on the phenomenon of digit force-to-position modulation is reviewed in Santello (2018).¹⁹

Figure 4. Grip devices used to study grasping at constrained and unconstrained contacts. The output of the 6D force/torque sensors was used to compute center of pressure of thumb and index fingertip. From: Fu, Zhang and Santello (1998). — **Figure 4.** Grip devices used to study grasping at constrained and unconstrained contacts. The output of the 6D force/torque sensors was used to compute center of pressure of thumb and index fingertip. From: Fu, Zhang and Santello (1998).

Figure 5. Trial-to-trial covariation of digit load force distribution (dLF) with vertical distance between digit center of pressure (CoP). The three centers of mass (left, center, and right) are color coded (red, black, and blue, respectively). From… — **Figure 5.** Trial-to-trial covariation of digit load force distribution (*dLF*) with vertical distance between digit center of pressure (CoP). The three centers of mass (left, center, and right) are color coded (red, black, and blue, respectively). From: Fu, Zhang and Santello (1998).

For more information on these projects contact: Marco Santello

References

Santello M, Flanders M, Soechting JF (1998). Postural synergies for tool use. Journal of Neuroscience.
Santello M, Flanders M, Soechting JF (2002). Patterns of hand motion during grasping and the influence of sensory guidance. Journal of Neuroscience.
Santello M, Soechting JF (2000). Force synergies for multifingered grasping. Experimental Brain Research.
Ciocarlie M, Goldfeder C, Allen P. Dimensionality reduction for hand-independent dexterous robotic grasping (2007). Proceedings of the 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems, San Diego, CA, USA, Oct 29 - Nov 2, 2007
Catalano, G., G. Grioli, E. Farnioli, A. Serio, C. Piazza, and A. Bicchi (2014). Adaptive synergies for the design and control of the Pisa/IIT SoftHand. International Journal of Robotics Research.
Santello M, Bianchi M, Gabiccini M, Ricciardi E, Salvietti G, Prattichizzo D, Ernst M, Moscatelli A, Jorntell H, Kappers A, Kyriakopoulos K, Albu Schaeffer A, Castellini C, Bicchi A (2016). Hand synergies: Integration of robotics and neuroscience for understanding the control of biological and artificial hands. Physics of Life Reviews.
Winges SA, Weber DA, Santello M (2003). The role of vision on hand pre-shaping during reach to grasp. Experimental Brain Research.
Rearick MP, Casares A, Santello M (2003). Task-dependent modulation of multi-digit force coordination patterns. Journal of Neurophysiology.
Rearick MP, Santello M (2002). Force synergies for multifingered grasping: Effect of predictability in object center of mass and handedness. Experimental Brain Research.
Lukos J, Ansuini C, Santello M (2007). Choice of contact points during multi-digit grasping: effect of predictability of object center of mass location. Journal of Neuroscience. Issue cover.
Lukos JR, Ansuini C, Santello M (2008). Anticipatory control of grasping: independence of sensorimotor memories for kinematics and kinetics. Journal of Neuroscience.
Lukos JR, Lee D, Poizner H, Santello M (2010). Anticipatory modulation of digit placement for grasp control is affected by Parkinson’s disease. Public Library of Science ONE.
Fu Q, Zhang W, Santello M (2010). Anticipatory planning and control of grasp positions and forces for dexterous two-digit manipulation. Journal of Neuroscience.
Fu Q, Hasan Z, Santello M (2011). Transfer of learned manipulation following changes in degrees of freedom. Journal of Neuroscience.
Fu Q, Santello M (2014). Coordination between digit forces and positions: interactions between anticipatory and feedback control. Journal of Neurophysiology.
Mojtahedi K, Fu Q, Santello M (2015). Extraction of time and frequency features from grip force rates during dexterous manipulation. IEEE Transactions on Biomedical Engineering.
Lukos JR, Choi JY, Santello M (2013). Grasping uncertainty: planning and execution of skilled manipulation for unpredictable object properties. Journal of Neurophysiology.
Marneweck M, Lee-Miller T, Santello M, Gordon AM (2016). Digit position and forces covary during anticipatory control of whole-hand manipulation. Frontiers in Human Neuroscience.
Santello M (2018). Dexterous manipulation: Understanding the continuum from hand kinematics to kinetics. In: Reach-to-grasp behavior. Brain, behavior, and modelling across the lifespan (Eds. Corbetta D, Santello M). Taylor and Francis Group.