“The people normally involved in human-computer intelligent interaction research are social scientists, psychologists, ethnographers—people who study the way people think and interact with their environments. And then computer scientists and engineers and people in art and design create the methods by which the interaction between the human and computers can happen,” Hasegawa-Johnson said.
By increasing campus interdisciplinarity, Hasegawa-Johnson believes that research projects will be enhanced, and Beckman will improve its international reputation in HCII. To help facilitate this, he is putting together a web site to improve communication between interested faculty members at Beckman and the university at large.
“What I love about being at Beckman is when I have ideas of things I want to work on that I have no prior knowledge of, I know there is someone in the building who is an expert in it,” he said. “Everyone is very helpful—if I approach them about a subject, at the very least will they will say, ‘This is what state-of-the-art is now, and here are a couple of things to read.’ And sometimes these conversations turn into year-long collaborations.”
Hasegawa-Johnson’s plan as co-chair of HCII is to expand those collaborations across campus.
“Beckman has a strong research history in social and computer science, but there are other avenues on campus we can tap into, like a partnership with art and design. I think we can improve the international prominence of our research by making it easier for people across campus to talk to one another about these subjects and, hopefully, encourage more grant proposals, papers, research, and educational programs.”
Dan Morrow serves as the other co-chair of HCII and is looking forward to increased collaboration across campus.
“It is exciting to have Mark join me as a co-chair of the HCII theme,” said Morrow. “Mark has a long history of working across disciplinary boundaries within Beckman and across campus. He brings a lot of energy to the position and has already made progress in helping further integrate human-computer intelligent interaction research at Beckman with other faculty on campus who have similar interests.”
Hasegawa-Johnson replaces Thomas Huang, professor of electrical and computer engineering. Huang was the longest-serving co-chair at the Beckman Institute.
As Hasegawa-Johnson steps into his new role as co-chair, he credits Huang for helping him find success at Beckman.
“Tom has been a great mentor to me,” Hasegawa-Johnson said. “He has helped me create the research program that I have. I owe a lot of my formative years here to the advice and support Tom gave me.”
I think we can improve the international prominence of our HCII research by making it easier for people across campus to talk to one another about [related research] and, hopefully, encourage more grant proposals, papers, research, and educational programs. — Mark Hasegawa-Johnson
After nearly 15 years of research at Beckman, Hasegawa-Johnson has built up an extensive program that embodies multidisciplinary research—he has research lines in a variety of subject areas and with faculty members in diverse disciplines.
He focuses on speech, audio, and language processing, with particular interest in technologies that can automatically extract information from speech, audio, or text data. He is also turning his future research toward interactive technologies, or systems that can respond to and learn from humans to make their lives easier.
“Many of the computer systems that interact with humans, like automated voice systems when you call a company for example, are too regulated and precise and are not replicated by anything you want to accomplish in the real world. I think we can make our human-computer interaction systems more robust by building systems that interact and learn from humans in real time,” said Hasegawa-Johnson. “This is a critical step in advancing speech technology. If these systems can learn in real time from human sources, then they will be much more effective and rapid in helping humans with whatever it is we want to do.”
One of the ways Hasegawa-Johnson is using interactive technologies is creating a system called Robo-Buddies, which employs pseudo-intelligent mediators that intend to improve communication between students with and students without physical disabilities. This project highlights the role technology can have in addressing the critical needs of diverse communicators in our societies, especially those with diagnosed communication disorders, by exploring ways to enhance people’s communicative competence and social participation both through modern communication and computerized technologies.
“We’re looking at ways a computer could help those with a communication disorder interact with their surroundings in a more productive and efficient manner,” Hasegawa-Johnson said. “My role is developing an interactive dialogue system through a variety of algorithms and human interactions that will then be transferred to an interface like a tablet so students can easily use it to communicate with others.”
This project, funded by a Focal Point grant from the University of Illinois, involves faculty from speech and hearing science, electrical and computer engineering, educational psychology, computer science, English, and special education—a truly multidisciplinary effort.
Hasegawa-Johnson is also building alliances with social scientists by providing speech technology support. He is working with Dorothy Espelage, professor of educational psychology, and M. Scott Poole, senior research scientist at the National Center for Supercomputing Applications (NCSA) and director of the Institute for Computing in the Humanities, Arts, and Social Sciences (I-CHASS), to study bullying in grade school playgrounds in order to develop a system that will automatically detect behaviors in groups.
“We have recordings of 40 children who put on microphones and played on a playground. Social scientists are going through the recordings and coding for aggressive actions, and then we’re creating a system that will automatically detect aggressive actions,” Hasegawa-Johnson said.
This is one scenario used to create a system called GroupScope, which will enable research into social interaction in large, dynamic groups to be conducted much more quickly and with much higher reliability than was previously possible. The system is created by automating many functions, including managing huge volumes of video, audio, and sensor data.
Many types of groups are being analyzed in this project, funded by the National Science Foundation (NSF), in hopes that GroupScope will shed light on the workings of critical functions performed by real world groups such as emergency response units, health care teams, stock exchanges, and military units. GroupScope will also have applications in the training of those working in multi-team systems, such as first responders to disasters. It can be used to record and “grade” training sessions, giving participants feedback on both strengths and weaknesses of their approaches.
Multidisciplinary projects like Robo-Buddies and GroupScope are the types of collaborative efforts Hasegawa-Johnson intends to help facilitate as HCII co-chair. Already, several faculty members from diverse areas such as industrial and enterprise systems engineering, advertising, and curriculum and instruction, have joined the Beckman Institute as affiliates in the HCII group, with intentions of starting collaborative projects.
“I’m excited about the great research we can all do together,” Hasegawa-Johnson said. “Increasing our collaborations across campus will strengthen our research and project outcomes, and this will, in turn, strengthen our international HCII presence.”
