Photos: Jeffrey Fitterman

Big Data for 3-D Hair Scan

Hao Li uses machine learning to build 3-D models of hair.

Creating digital human beings is the stuff of big, blockbuster movies, but soon the power to create your own highly realistic digital double will be in your hands, thanks in part to researchers like Hao Li, assistant professor in the Department of Computer Science. His interest in the future of computer graphics, virtual reality and augmented reality have led him to digitizing people in real time. Right now, he’s working on hair.

Hao Li
Hao Li

“Hair is one of the hardest things to model in computer graphics,” Li said. In movies with complex hair like Brave or Frozen, artists have to define every strand and each curl, and that’s just the shape. But hair is one of the basic features about us that makes us look like who we are, so it’s important to get it right.

Li’s lab first worked on algorithms that could render a 3-D model of loose, unconstrained hair. Now, he and his team have moved onto more complex situations for hair capture: braids.

Li and his team watched YouTube video tutorials to learn how to do all different styles of braids — simple, French, Dutch, fishtail, waterfall, and more.

The variation in braid styles is only part of the difficulty in modeling them. They can be all different thicknesses and levels of neatness. For the computer to automatically render a digital braid based on input photos, it needs to extract a few properties and then compare them to a database of braid possibilities to find the closest match.

At first, Li and his team captured images of these braids with 50 cameras suspended from scaffolding surrounding the subject, but Li streamlined the process so it could be done with simpler hardware: a single Xbox Kinect. It uses 2-D orientation field analysis, looking at the hair and seeing how it flows. Li’s algorithms also work independently of how the captured images are lit, since that can vary so much.

Next, Li wants to generate 3-D models of hair from smartphone selfies — even those where part of the person’s hair is obstructed or out of frame. Soon, with just a smartphone, everyone will be able to construct personal 3-D digital doubles.

“The big impact would be for an ordinary person to have access to these high-quality 3-D models to create an avatar of themselves they could play a game with,” said Li.