Skip to main content

Surface Haptic Interaction Design


Surface haptic interaction design focuses on the perceptions and experiences of the user during tactile interactions.  We study topics ranging from fundamental principles underlying multi-finger tactile perception to haptic display techniques which can improve safety in automobiles or user experiences with personal electronic devices.

Automotive Interaction

Once the domain of purely physical controls such as knobs, levers, buttons, and sliders, the vehicle dash is rapidly transforming to include touchscreen interfaces. This presents a challenge for drivers, because the physics-based cues which make traditional controls easy to operate with limited visual confirmation are absent on traditional screens. A collaboration with Ford Motor Company, this research investigates the addition of surface haptic feedback to a visual display. 

Multi-Finger Sensory Fusion

Automotive InteractionThe goal of this project is to better understand the perceptual mechanisms involved in the translation of raw sensory cues from multiple fingers into rich descriptions of the physical world.  On a surface haptic device, force profiles which vary in space and time can be used to display a variety of virtual features to a single finger.  These include edges, elastic and viscous regions, out-of-plane contours, etc.

However the use of multiple fingers introduces additional possibilities including graspable interactive objects with inferred properties such as inertia and shape.  Furthermore, we find that even simple haptic features such as bumps and holes elicit higher level mental models when explored by multiple fingers.  For instance, we have discovered that contour information gathered from one finger can be modulated by simultaneous information from other fingers and in some cases can even be perceptually referred to a new location separate from either finger.

Multi-finger sensory manipulationIn order to conduct human subject studies on these and other effects without the size and force constraints of the haptic surface technologies currently under development, we design specialized high-fidelity mechatronic devices to deliver the necessary planar forces.  The aim is to develop our understanding of the tactile perceptual model in parallel with the vibratory and electrostatic surface haptic device development.

TPaD Tablet

The TPaD Tablet Project is an effort to get our variable friction devices (TPaDs) to a broader community of human computer interaction (HCI) researchers and interaction designers so that they may exploit these new capabilities in their own work. The TPaD Fire combines an Amazon Kindle Fire with a TPaD, and aims to deliver a unique interaction environment that is low-cost and easy to use. Visit the project page or contact Joe Mullenbach for more information. 


Edward Colgate
Michael Peshkin
Joe Mullenbach
Craig Shultz
Steven Manuel

Related publications

Jin, S., J. Mullenbach, C. Shultz, J. E. Colgate, and A. Marie Piper, "OS-Level Surface Haptics for Touch-Screen Accessibility", ASSETS, Rochester, NY, 2014. Google Scholar | BibTex | RTF

Mullenbach, J., C. Shultz, J. E. Colgate, and A. Marie Piper, "Exploring affective communication through variable-friction surface haptics", Proceedings of the 32nd annual ACM conference on Human factors in computing systems - CHI '14, Toronto, Ontario, Canada, ACM Press, 2014. Google Scholar 

Mullenbach, J., C. Shultz, A. Marie Piper, M. A. Peshkin, and J. E. Colgate, "TPad Fire: Surface Haptic Tablet", HAID Haptic and Audio Interaction Design, Daejeon, Korea, Springer, 04/2013. Google Scholar 

Back to top