Konstantinos Bousmalis ’05 among 34 Google Fellowship Recipients Worldwide

By Samuel T. Clover ’91

Picture this: After a long day at work, you sit down at your computer, shoulders hunched, and a digitized voice coos, “You seem stressed today.” Suddenly, peaceful music starts playing with photos of secluded beaches and desolate mountain passes on your screen. Your shoulders relax; you breather easier. Your computer—that tidy assemblage of plastic and silicon—has “read” you, and responded.

This isn’t HAL9000, the learned computer from 2001: A Space Odyssey that famously went rogue. In fact, it isn’t any computer at all—yet. But if Konstantinos Bousmalis ’05 and his research team at Imperial College London are successful, computers will soon be able to determine users’ emotions by analyzing body gestures and nonverbal cues, and react accordingly.

For his work, Bousmalis was one of only 34 research students worldwide to receive a  prestigous Google Ph.D. Fellowship, which provides full tuition and a stipend.

Bousmalis credits Lafayette’s EXCEL Scholars program for his start in research. Though he had several mentors, he became especially close to Jeffrey Pfaffmann, associate professor of computer science, who had just arrived on campus. Bousmalis published his first paper—on evolutionary computation techniques—with him, and continued that research at Edinburgh.

Konstantinos Bousmalis '05 (right) with Ph.D. adviser Maja Pantic (far left) and Alessandro Vinciarelli, lecturer in computing science, University of Glasgow, at workshop on predictive models of human communication dynamics, Los Angeles, Calif.

A computer science graduate, Bousmalis grew up in Thessaloniki, Greece. He earned an M.Sc. in artificial intelligence at University of Edinburgh before pursuing a Ph.D. in computing at Imperial. There, he’s part of iBUG, the Intelligent Behaviour Understanding Group. Led by Maja Pantic, Bousmalis’ adviser, the international cadre of scholars enables “machines” to analyze facial gestures, body movements, and human visual and audio cues. Pantic is professor of affective and behavioural computing.

“My Ph.D. is on social signal processing,” Bousmalis says. “This is a combination of machine learning and social psychology with the goal of bringing social intelligence to computers. The idea is to be able to understand not just emotions—happiness, sadness, surprise—but also states of mind, what we call social signals, which can provide information about interactions, social emotions, social actions, or social relations.”

The computer “learns” how to identify emotions by analyzing body gestures revealed in long series of photos and videos.

“Let’s say you have 100 videos, or clips, of people raising their eyebrows compared to 1,000 clips of people not raising their eyebrows,” Bousmalis explains. “Machine learning comes in and you attempt to create computational models that are able to learn this distinction by extracting features from these videos and trying to see what happens in each case. It’s like teaching a kid through examples, in a sense.”

Of course, this kind of analysis can get complicated because people are complicated—especially people such as politicians.

“For example, you’ll have a video of a political debate, which is part of what we’re doing,” Bousmalis explains. “You try to analyze a face. Was there a smile, a frown? Was there a frown with a raise of pitch in the voice? Was there a movement of the hand in a particular way? Was there a head nod at a particular point, a head shake? We’re trying to collect all the information that an audiovisual stream will give us.”

Bousmalis attended an American high school in Thessaloniki, where most courses are taught in Greek. He says that his family could not have afforded “a full-blown U.S. education” without Lafayette’s generous financial aid.

And, he credits the International Students Association for giving him “tremendous support,” and helping him forge several long-lasting friendships.

“I have a lot of really good friends from Lafayette, not all of them international students,” he says. “We still keep in touch and meet up, even though we live thousands of miles apart. When one friend got married, we had a bachelor trip in Greece. People came from China and San Francisco, Philadelphia, New York, Paris, London. It was fantastic.”

Those are the kinds of friendships that computers, no matter how intelligent, simply can’t replace.