On a visit to Heather Culbertson’s lab, you might be gently patted on the shoulder by a robot. How would that make you feel? If the answer is “a bit uncomfortable,” you are not alone.

“A lot of people have never been touched by or interacted with a robot, so at first, they’re a bit weirded out by it,” said Culbertson, a USC Viterbi assistant professor of computer science.

After noticing this tendency, Culbertson and her students programmed the robot to kick off the interaction with a friendly high-five. It worked — people loosened up and were more willing to interact with the robot going forward.

Welcome to USC’s Haptics Robotic and Virtual Interaction Lab, where Culbertson and her students explore how humans interact with the world, robots and technology through touch.

This experiment is part of a study exploring touch in assistive robots for the elderly and people with disabilities. The intent is not to replace human caregivers, stressed Culbertson, but to understand whether people want in-home robots as companions as well as helpers.

“If an assistive robot helps someone to get out of bed, get dressed or brush their teeth, we want to know if people also want to interact with these robots socially,” said Culbertson. “Or do they just want the robot to do its job and focus on the robotics?”

 

Touchable Technology

It’s one of many projects underway in Culbertson’s lab, all of which focus on what’s arguably the most underappreciated human sense: touch. Culbertson’s research — the lab’s name comes from the Greek word “haptikos,” which means sense of touch — spans wearable devices, virtual reality, medicine and human-robot interaction.

“You can convey and receive so much information through touch, yet it’s a much-underused sense in technology today,” said Culbertson, who joined USC from Stanford University in 2017.

“People have been focusing so much on visual and auditory senses, but we don’t communicate through vision and hearing alone. In the digital world, we’ve lost that tactile sense. How can we bring some of that back into our online experiences?”

In the past, haptics has been used in somewhat unsophisticated applications, such as vibration alerts on cellphones or rumbles from video game controllers. By contrast, in Culbertson’s lab, touch is used to create a panoply of sophisticated sensations for applications ranging from entertainment to therapeutic purposes.

For instance, one device in the prototype stage in Culbertson’s lab uses the principles of touch therapy and virtual reality to help people overcome their fear of public speaking. Another, a wearable armband, creates the sensation of a pleasant, calming stroke on the arm and could allow people to communicate through touch over long distances.

 

Tricking the Brain

Culbertson, a mechanical engineering major, grew up “building things.” Her goal is to bring the sense of touch to objects in virtual reality, realistically mimicking touch sensations experienced during interactions with the physical world.

“People want realism,” said Culbertson. “They want it to actually feel like a button click. It should feel like I’m actually picking up this object and interacting with this material, not just like an approximation of it.”

 Though not without challenges, realistic haptic feedback is possible, in part, because the brain is easier to trick than you might expect.

The sense of touch isn’t perfect — it relies on mechanical signals that your brain combines in order to tell you what’s happening in the physical world.

 Culbertson is capitalizing on this tendency for the brain to be fooled by creating realistic textures in virtual reality, so not only do objects feel solid, they feel as if they’re made of very specific materials.

 

Data-Driven Haptics

To create the sensation of a texture, Culbertson uses a method called “data-driven haptics.” First she records what happens when an object such as a finger is dragged over fabrics and other materials at different speeds and pressures. From that, she creates a mathematical model.

Then the vibrations and sounds associated with a particular material are artificially generated and “played” when a user drags a digital pen across a tablet surface, making it feel like they are touching a specific textured object — say, smooth silk or scratchy wood.

Imagine being able to feel the material of your next sofa before you buy it, or turning the “pages” of your next e-book. In medical training applications, the technology could allow surgeons to “feel” different bones and tissues as they cut and suture patients that exist only in virtual reality

If the current wave of haptic innovation is any measure, reaching Culbertson’s goal of integrating touch into the digital world is only a matter of computer processing power and time.

“I think in the future, people will want a realism in our digital lives that lends a sense of immersion,” Culbertson said.

“We have to push the envelope if we want to change the way humans interact with machines. I think haptics will be one of the few technologies we’ll see in our lifetime that fundamentally changes computing.”