Understanding Language as it Happens

Sarah Brown-Schmidt believes that in order to understand language production, a researcher has to get in on the conversation – in a way that replicates how it happens in real life. In order to do that, the Beckman Institute researcher creates experiments that focus on how it is language is produced when people are actually talking to one another.

“In psycholinguistics there is a long history in studying language use in pretty artificial settings in which (test subjects) are not actually talking to another person,” Brown-Schmidt said. “For me that is a less interesting avenue of inquiry because it’s not going to be representative potentially of the way language is used normally. I prefer to start from asking questions from normal conversation settings.”

That approach has led Brown-Schmidt to incorporate eye-tracking equipment in her research, going back to her days as a graduate student and while earning a doctorate in Brain and Cognitive Sciences at the University of Rochester. The use of a technology that is unobtrusive and that doesn’t require interaction is an integral part of her research into psycholinguistics.

“I’m interested in the way you are using language, how your mind understands it, and the brain processes that allow you to speak,” Brown-Schmidt said. “So what is it mentally that you are doing that allows you to talk to me in a conversation.

“It’s a belief that if you’re interested in any kind of psychological or mental process, that the best way to study it is to study it in a way that people normally use that process.”

“I’m interested in the way you are using language, how your mind understands it, and the brain processes that allow you to speak. … It’s a belief that if you’re interested in any kind of psychological or mental process, that the best way to study it is to study it in a way that people normally use that process."
– Sarah Brown-Schmidt

Brown-Schmidt is an Assistant Professor of Psychology at the University of Illinois who first came here as a postdoctoral researcher in the area of language production. She later earned a Beckman Postdoctoral Fellows appointment and is now a faculty member in the Cognitive Science group at the Institute. Brown-Schmidt is a native of Seattle who found her academic muse at Reed College in Oregon when she took a class in psycholinguistics.

“I went to college thinking I was going to go into political science,” she said. “I had to take a psycholinguistics class and found myself asking 50 million questions.

“So I thought ‘this is a good indicator of my interests,” she added with a laugh. “It was really compelling and super interesting. I just knew it from then on.”

After graduation, Brown-Schmidt worked in labs where eye-tracking equipment was standard. She adopted the method, which she says has been used in the field for only about 15 years, for her work studying the mechanisms by which, she writes, “people produce and understand utterances during the most basic form of language use: interactive conversation.”

Brown-Schmidt said that having test subjects interact with a computer – such as giving responses to stimuli on a screen – allows experimental control when it comes to studying word frequency, “but it doesn’t necessarily scale up to how words are used when you are actually talking to somebody.

“There’s a separate field coming out of linguistics looking at these interactive situations,” she added. “But they’ve lacked the experimental methodology to look at things like the time scale at which words are understood, so they are more descriptive. My work bridges those two domains, taking the really rigorous methodologies developed by the psychologists with the more natural, interactive methods developed in studies of discourse and linguistics, and methods using eye-trackers.”

Brown-Schmidt’s approach has already made an impact, beginning with her finding that in conversational settings people use unclear or ambiguous language a lot more often than would be expected. She said that in experiments using games such as Lego blocks, for example, they found that a listener understood which block their partner was talking about even with ambiguous language because of other input that takes place during conversation.

“If you are in this interactive setting and playing with Lego blocks, a lot of times the listener is not confused because the way you are playing with them narrows down what you are talking about,” she said.  

The ability to employ eye-tracking (she uses both table top and head-mounted equipment) enabled these and other findings, Brown-Schmidt said.

“I have a really nice measure of attention and can use it to measure people understanding each other in a way that doesn’t require pushing a button every time they understand a word the other person says,” Brown-Schmidt said.

In her research statement, Brown-Schmidt writes that she studies topics such as how message formulation processes interact with utterance planning processes, what information is represented about a conversational partner, and the time-scales at which these representations are used by the language processing system. One of her main research topics as a doctoral student that continues to this day is what she describes as “your ability to remember or represent what others know.

“It’s where someone has some private knowledge about something that the person they are speaking with doesn’t know about. So, how do I take what I know you know and use that to help me understand what you’re trying to say to me.”

Brown-Schmidt said that work has been extended into a recent collaboration with Beckman colleague Neal Cohen looking at brain and memory contributions to language processing by studying those processes in people with amnesia. The research has potential clinical relevancy, especially for those with declarative memory impairment, and possibly for individuals with Alzheimer’s disease.”

“For any individual who has severe declarative memory impairment, our findings suggest there is going to be a lot of difficulty in tracking the continuity of a conversation over time, particularly in cases where you are interrupted,” Brown-Schmidt said. “It looks like, unfortunately, that the record of the conservation disappears quite quickly with distraction.”

Brown-Schmidt also collaborates with Beckman colleagues Annie Tremblay (on a project that studies how English speakers understand people with a French accent) and with Duane Watson, her fellow Cognitive Science group member and husband. Brown-Schmidt first got interested in Beckman because of a research focus on language processing, an area of strength at Beckman due to researchers like Kay Bock.

Watson joined the Psychology Department faculty in 2005 when she became a postdoctoral researcher; she later became a Beckman Fellow and then moved into a faculty position. Brown-Schmidt said the dual researcher/faculty household works well for her and her husband.

“It’s fun. For both of us the science is just really exciting,” she said. “We’re both passionate about it and we think about it all the time. We’ll be on a car trip or whatever and we’ll be like ‘oh isn’t it interesting the way person’s accent is’ and have a fun conversation about it.”

Although, Brown-Schmidt says, studying conversations and engaging in them are two different things for her.

“Sometimes, you study the things that you’re worst at,” she said. “Probably a lot of people would think of me as a very talkative person but I think of myself as very shy.

“Nobody,” she added with a laugh, “would agree with me.”