Teaching a Computer to Recognize Bill Clinton

USC’s Information Sciences Institute leads the way in face recognition -— a technology with huge implications for security, commerce and even recognizing what’s-his-name at your high school reunion

Screen Shot 2016-03-30 at 12.30.19 PM

Six pictures of Bill Clinton. One shows his upturned face partially obscured by Pope John Paul II. Another shows him older and from another angle as he enthusiastically clasps hands with supporters.

There’s a staged portrait of him and one of him kicking a soccer ball.

For computers, however, recognizing the same person in pictures with variations in lighting, pose, age and facial expression has been downright confounding.

Until recently.

It’s clear it’s Clinton in each photo. For cutting-edge research conducted over the past two years, a multinational team of scientists led by USC’s Information Sciences Institute (ISI) has been chosen by the federal government as one of the best in the world at teaching computers how to recognize people “in the wild,” that is, through photos taken in uncontrolled settings, with a multitude of variations in age, pose, etc.

Using new algorithms and software, and working with an initial data set of 500 highly photographed people like Bill Clinton and actor Harrison Ford, the team has “taught” computers how to identify a person even if part of his or her face is obscured or seen from different angles or in varying degrees of clarity.

The possibilities for accurate face recognition are endless, with security-related uses most obvious, said Tal Hassner, a senior computer scientist at ISI and a professor in the Department of Mathematics and Computer Science at the Open University of Israel.

Beyond that, think of those social media games that claim to tell you what Disney character you most closely resemble, cars that won’t start unless a computer in the dashboard recognizes the driver, or even computerized eyeglasses that will tell you the name of the person you don’t recall who just greeted you, Hassner said.

“Computers have an infinite capacity to process lots of information, and a bit is a bit — it’s either a zero or a one,” said Hassner, who teaches computers how to understand the world through the visual input of photos and videos. “An image, to a computer, is just a huge matrix of pixels, and to recognize a person in a picture, it needs to compare values of numbers located in one place with values in another.”

Just two years ago, two of the best commercially available computer systems were able to deliver only a 50 percent face-recognition accuracy “in the wild,” said Wael Abd-Almageed, a senior scientist at ISI who is the technical lead on ISI’s face recognition project, dubbed GLAIVE (it’s an acronym, but also the French word for a single-edge blade on the end of a pole).

Wael Abd-Almageed and Tal Hassner
Wael Abd-Almageed and Tal Hassner

Now, competing against three other teams on a federal initiative for the Office of the Director of National Intelligence, the ISI led project has achieved a high-performing 89 percent success rate, according to the Abd-Almageed.

“We’re not exactly teaching computers how to think like humans,” he said, “but to mimic one of the human functions. Humans are really good at recognizing faces. I see you now, I see you in couple of days and you might have a beard or sunglasses or be wearing different clothing, but I will still recognize you. This has been really difficult for computers to do.”

One of the ways ISI scientists are teaching computers how to recognize faces is by extracting facial “landmarks.” In ISI’s system, there are 68 — eyebrows, nose, mouth, etc. The system uses these landmarks to digitally rotate the face, say, from a profile to head-on, a process called frontalization.

Using code and algorithms, the GLAIVE project also is teaching computers to mimic how neurons in the brain talk to each other. The big question is, if a computer is trained to mimic how the human brain works, would it be able to recognize a new picture of someone that already is in its data bank with human-like accuracy?

“That actually might occur sooner than anyone expected two years ago when we started this project,” Abd-Almageed said. “We have pushed the envelope significantly in the last two years.”

The ISI team is nearing the end of the first phase of a three-phase project that should last another two-plus years, Abd-Almageed said. The next data set of images the team will analyze will be of 2,000 people, with multiple images of the same person, while the program’s accuracy requirements will be 10 times smaller.

Abd-Almageed said upcoming facialrecognition software likely could be used by the government to improve security, say, at airports. However, he and his team aren’t focused on how it will be applied.

“Our job,” he said, “is to get it to work.”