Brandon Smith likes solving problems, and that’s why his new position at the Beckman Institute’s Illinois Simulator Laboratory (ISL) as a simulation systems research programmer is a perfect fit. 

“The best part of the job is the creativity and freedom I’m given. I’m told what problem to solve, not how to solve it, and I can just use my own knowledge of knowing what’s out there, researching the options, and making my own decision about the best solution,” Smith said. “I didn’t know there was a job like this, where I could just creatively explore all these avenues.”

In his few months on the job, Smith has worked on several different projects involving the simulation hardware at the ISL. The lab’s resources are open to any researcher worldwide, and Smith assists researchers in finding the best way to incorporate the ISL’s hardware capabilities. Housed on the south end of campus, the ISL specializes in facilitating advanced scientific understanding of human-computer interaction, and features a flight simulator, driving simulator, motion capture suite, and the Cube and Cave, two immersive 3D virtual environments.

His first project was working with Lisa Frank, former Beckman Senior Fellow, and Hank Kaczmarski, ISL director, to develop a 3D immersive experience in natural and urban environments to ultimately test if virtually walking through the environments would positively or negatively affect those with post-traumatic stress disorder (PTSD). He developed this environment with the next-generation Cube, which uses three 90-inch, LCD/LED monitors. 

Smith’s job was to program the video player so that the virtual environments played across multiple screens, side by side, and then sync it with the treadmill, so that when participants walked, the video followed along with their footsteps. 

Smith is also working with veterinary medicine researcher Janet Sinn-Hanlon and Kaczmarski to create 3D interactive models of Digital Imaging and Communications in Medicine (DICOM) data, which includes images from MRI, CT, and PET scans, so that veterinary medicine researchers can virtually dissect and interact with DICOM scans, like the MRI scan of an animal leg.

The Cube is connected to an ordinary Xbox Kinect game system, which utilizes movements from the human body to control the screen. A Kinect doesn’t require a controller—a camera built into the system picks up body movements so users can interact with the content on the screen in a natural way. Researchers can use one arm to grab a piece of the image, and another arm to move, rotate, and zoom in and out. 

“I wanted to build a system that would be intuitive to researchers, so they wouldn’t have to use any controllers—they could just use their body. So the Kinect system made sense,” Smith said. “Researchers can use their arm movements to make cuts, dissect, and fly these muscles, tendons, and bones around in the 3D environment.” 

In the future, researchers would like to use this system for training programs so veterinary medicine students can virtually conduct surgery without the use of a cadaver. Long-term goals are to allow medical professionals to view 3D MRI data before a surgery, for instance, so they are better prepared for a procedure. 

Smith received both his undergraduate degree in statistics and computer science and his master’s degree in bioinformatics and animal science at the University of Illinois at Urbana-Champaign. After working as an independent video game creator and then as a programmer for an educational game company, Smith decided to look into coming back to the U of I. 

“This job was just too good to pass up—not only did it really interest me, but it fit my skills really well,” Smith said.

One of the things Smith would like to bring to the ISL simulation software is graphics from the Unity Game Engine, a game development ecosystem that offers high-quality rendering of 3D and 2D content. This system will allow for new kinds of research scenarios to explore, yielding the greatest possible advances in understanding the connections between humans and technology. 

“That’s what we like to do at the ISL,” Smith said. “Bring in our cool technology and then help people out.”