Lansing's Research Shows Importance of Visual Cues in Communication

From studying why cell phones annoy bystanders to improving hearing aid technology, Charissa Lansing's research has focused on the importance of visual information to human communication.

Translating basic scientific research into real-world applications influenced Charissa Lansing's academic career while earning a Ph.D. in speech and hearing science and continues to guide her work today as a Beckman Institute faculty member.

"That was what drove my idea of my career and what I wanted to accomplish: the opportunity to take information from research and translate it to an application," Lansing said.

These days, Lansing is investigating a topic that's as real world as the evening news: cell phone users and the people who are annoyed by them.

Cell phones have become to manners what half-filled glasses of water are to optimists and pessimists. There appears to be no middle ground when it comes to the ubiquitous little devices that have half the population walking around with one hand stuck to their head, talking out loud in a way that used to get people locked up.

If you're a "glass is half-full" person who thinks cell phones are a wonderful innovation that enables communication almost anywhere, Lansing's research has something to quiet your critics. And if you are one of those critics who think all cell phones should be confiscated and crushed into salvage-yard bundles, her work may offer hope for a less-intrusive future.

"This might be a real opportunity to use computerized agents like avatars or animations to help provide some visual cues that will mediate some of the communication."
- Charissa Lansing

Lansing is a faculty member in the Human Perception and Performance group at Beckman who studies how people select and then integrate visual and auditory information in order to process language. The cell phone project, funded by wireless technology developer Qualcomm, looks at the social interactions of cell phones users and bystanders, including issues such as why cell phone users annoy those around them.

"The popular perception is that the speech on cell phones is annoying, so we wanted to try and figure out this whole problem of annoyance and try to better understand it," Lansing said. "If we could understand some of the factors that create the annoyance or contribute to the annoyance, there may be some technological ways to solve some of those problems."

Lansing is the principal investigator for the cell phone study, but it is just one of many projects for the Associate Professor of Speech and Hearing Science at the University of Illinois at Urbana. Lansing was a key part of the Intelligent Hearing Aid project along with Beckman colleagues Al Feng, Bruce Wheeler, Doug Jones, and William O'Brien. These days she has several interdisciplinary projects under way.

The study looking at cell phone annoyance and the psychophysics of their use is a multi-year project. As part of her responsibilities as PI of the project, she is designing experiments for the study.

Lansing and her collaborators began with an experiment that had the test subjects perform tasks such as mental arithmetic and proofreading while in the presence of overheard one-way cell-phone and two-way face-to-face conversations. Lansing said the experiment controlled for things such as overall voice volume, amount of talking time, conversational topic, and tone of speech.

"The outcome of this experiment was that people were no more affected by the two-way conversation than they were by the one-way cell phone conversation, and they were no more frustrated in terms of their ratings on the scale between the two conversations," Lansing said. "It doesn't interfere with your ability to be accurate in those tasks or to become frustrated any more than when it's two people that you're in the presence of their conversation, or one person talking.

"So that alone didn't explain part of this large annoyance that the population has with cell phones."

So, much like the cell phone study at Beckman that showed talking on a cell phone is distracting to drivers but no more or less so than other tasks, this project has so far gone against conventional wisdom. Cell phone users may not be any more annoying than other people carrying on a conversation, but the study does show that people doing tasks can get annoyed by them.

"When it happened, we certainly had some (test subjects) who were visibly annoyed and let us know about it," Lansing said. "Some people got up and got out of the room and told us to stop talking, that it was bothering them. Others, if they were doing the proofreading, started reading aloud in a louder voice."

In a separate set of pilot studies, Lansing and her collaborators demonstrated that talkers on cell phones speak more loudly than they do in face-to-face conversations. Lansing said the cell phone problem likely has to do with visual cues that people use while communicating. People talking on a cell phone don't get those visual cues, so they may feel the need to speak more loudly or repeat themselves more than they would in a face-to-face conversation.

"Our next project is going to look at how just having visual cues actually mediates the visual exchange," Lansing said. "We know that when there are visual cues present, even if it's just seeing the other person, that there's less need to repeat things; things like emotional expression and your intent may be better understood because there are more cues there."

Solutions for less intrusive cell phone use could in the future include video phones. But those devices require high bandwidth in order to transmit the kind of high-resolution images compelling enough to provide visual cues. One remedy could be computer-generated avatars that require less megabytes. Lansing's HPP colleague and collaborator on another project Jesse Spencer-Smith has developed just such an avatar.

"This might be a real opportunity to use computerized agents like avatars or animations to help provide some visual cues that will mediate some of the communication," Lansing said.

The importance of visual cues in communication is a key aspect of Lansing's work.

"I would characterize my interest as the area of auditory-visual integration, how visual cues contribute to speech understanding," she said.

Other projects for Lansing include a collaboration with Beckman colleague Susan Garnsey looking at representations in the brain of auditory and visual information and how they are integrated. Another research line has Lansing studying how communication occurs when American Sign Language is used simultaneously with facial expressions and lip movements in spoken English. The findings provided evidence of differences in visual processes related to the location of critical information, sender characteristics, and among groups of deaf versus hearing adults. That work resulted in a paper, Eye Fixations of Deaf and Hearing Observers in Simultaneous Communication Perception that appeared in the August 2006 issue of the Journal of Ear & Hearing.

That project was innovative in relying on eye-tracking technology, an important component of Lansing's research work. After getting her Ph.D. at the University of Iowa, Lansing came to Illinois and integrated new technologies into her work with the help of people like eye-tracking pioneer and longtime Beckman researcher George McConkie.

"When I first came to the University, one of the first people that I sought out was George McConkie," Lansing said. "I wanted to learn about some of the tools that could be used to test my hypotheses about understanding spoken language through the use of visual cues. It was wonderful to work with him and to use new technology such as eye-tracking."

The adaptation of eye-tracking technology to the discipline of speech and hearing science has been one of her biggest contributions, Lansing said.

One contribution has been to show that technology such as eye-tracking is a feasible way to test hypotheses about audio-visual speech perception," Lansing said. "It's been something that has not had a history in speech perception. It has been used to look at other kinds of cognitive processing, but it has not been used to look at speech understanding until more recently."

Lansing lists the Intelligent Hearing Aid project as one she is proud to have worked on.

"That experience, working with the group that we had, was such a wonderful model of collaborative research, of interdisciplinary research," she said. "It was a tremendous opportunity to engage students at every level of their career. It was such a stimulating, exciting experience."

Lansing began her professional career as a speech and hearing clinician. The desire to add research-based scientific knowledge to the field was a big motivating factor in her earning a Ph.D. That desire still motivates her today, no matter the topic. The fact that she is involved in highly interdisciplinary projects that involve researchers from fields like computer and electrical engineering and psychology show the science-based contributions speech and hearing science can make to a wide range of research projects.

"I think people in speech and hearing have a real opportunity to play an integral role for the campus in general in its mission of translational research," Lansing said. "We can bring to the table the combination of applied research with how that relates specifically to people who have communication disabilities."