A decade ago Steve Sullivan was a research assistant at the Beckman Institute on the lookout for job opportunities when he decided against a career in academia or one of the traditional industry outlets for an engineer with a brand new Ph.D. Instead, he chose to go to Hollywood.
As an electrical engineer, Sullivan’s dreams of making it in the movie business differed from the standard Hollywood formula. Sullivan wanted to work behind the camera and knew the computer vision techniques he focused on at Beckman could provide real- world solutions to moviemakers who in the 1990s were grappling with how to apply new digital technologies to some of the problems inherent in traditional filmmaking.
But like many a Tinsel Town dreamer, Sullivan was hit with a spark of inspiration emanating from the screen. He was wa t c hing a TV documentary about the making of Jurassic Park and instantly realized that computer vision techniques could replace the laborious and restrictive camera location methods the show was describing for creating special effects.
“It was talking about how movies have this problem of matchmoving and figuring out where the cameras were and objects were and that is computer vision,” Sullivan said.
Sullivan began talking to people in the industry and telling them about computer vision techniques that could solve special effects problems. In 1996 he landed a job with a graphics company in Los Angeles called Rhythm and Hues that did work for the movies.
“So that’s how I got into the industry.” Sullivan said. “I was applying exactly those things that had I wo r ked on at Beckman.”
But his dream job was to work at George Lucas’ famed special effects company Industrial Light and Magic, located in northern California. That desire was realized in 1998 and since then the Beckman alumnus has been, well, living large: winner of an Academy Award for technical achievement, soon-to-be White House honoree, and, oh yes, recipient of a kiss from actress Charlize Theron.
Not the usual perks for someone with a Ph.D. in Electrical and Computer Engineering from the University of Illinois. But then Sullivan is not your typical electrical engineer, according to Jean Ponce of the Artificial Intelligence group. Ponce served as teacher, academic advisor and mentor to Sulliva n when he worked in Ponce’s Computer Vision and Robotics group at Beckman.
“He was this kid from Missouri with a funny haircut. But he wanted to do something with himself,” Ponce said.
Ponce said ambition wasn’t the only thing that separated Sullivan from his peers.
“Steve was an interesting guy, and a fun person as well,” he said. “He had a gift for person-to-person relationships that I think is rare for a Ph.D. student because they focus on their work. But Steve had other interests.”
Sullivan’s early graduate studies were in areas like medical imaging and robotics before later focusing on topics such as image-based modeling and 3-D tracking that would play a role in his future career. He joined Industrial Light and Magic just as the movie industry was looking to incorporate the kind of computer vision techniques he learned as a research assistant for Ponce.
At that time, computers had widely been used to create special effects. But the mov i emaking process was still hampered by the age-old methods of calculating camera angles and positions by hand, while filming the effects separately from the rest of the film and then integrating them, sometimes clumsily, into the movie.
Sullivan was a pioneer in writing algorithms for computer-vision software for automatic camera-tracking. The computer vision matchmoving method measures two- dimensional and 3-D objects in images (a process called photogrammertry), and then reconstructs them for integration with special effects. The technique is a more efficient and robust process for creating special effects and gives them a much more realistic look than before.
Today, computer vision methods like matchmoving and photogrammertry have become the standard as special effects and moviemaking in general underwent a revolutionary change in the last few years with the advent of digital technology. Sullivan said computer vision frees filmmakers to shoot the movie they want because the special effects can be integrated seamlessly once filming is done.
“It changed the kinds of stories that can have effects in them,” Sullivan said. “They can take it for granted now that you can put effects anywhere you want to put them.”
In 2001, Sullivan won an Academy Award in Technical Achievement for development of the ILM Motion and Structure Recovery System (MARS) and two years later, he was named Director of Research and Development at ILM. Sullivan and ILM continue to break new ground in movie special effects. One of their major goals is to create the first completely digital, realistic human actor.
“We do basic R&D to make sure that’s going to be possible when the movie comes along,” Sullivan said, “but you won’t see that come out on the screen until the right movie comes along.”
Sullivan said another big project, in concert with the LucasArts video game company, is to develop a virtual reality, game-based method of collaborative moviemaking. Sullivan said this new project could be as important as computer vision in changing the way movies are made. The program would allow writers, directors and other players to engage in virtual storyboard sessions that could let them know, among other things, exact camera shots and positions prior to shooting the scenes.
“This is really about tools for the director and tools for the artist to work collaboratively with each other, rather than assemblyline fashion, which is what they do now,” Sullivan said. “The same way you might have a multi-player video game where you have characters running around in this computer graphics world and you have a certain viewpoint on that, a director could work in that style to make their movie.”
Changing the way movies are made and contributing to special effects blockbusters like Minority Report and the Star Wars saga is a dream career for many. But Sullivan has some very down-to-earth advice for those students who would like to follow in his footsteps.
“First, take as much math as you can possibly get,” Sullivan said. “You’ll never regret that. It’s very easy to learn programs or new systems when you’re out in the workforce but very, very difficult to learn new math on the job.”
Sullivan also says that anyone interested in working on computer games should become part of the development process.
“Get an evaluation copy or buy a license for one of these 3-D packages and try making visual effects on your own,” he said. “You have to try it to really understand it.”
Thirdly, he said, “look for applications outside of what you know.
“I’m the poster child for that. I had no idea this would apply to visual effects but just by paying attention to what other fields’ problems are, even if you don’t know the solution, it’s a great way to do something that’s brand new.“
Sullivan said his time at Beckman paid off in that regard.
“That was what was really great about Beckman is you were exposed to all these different disciplines that may have, on the surface, very little to do with each other,” he said. “But down at the bottom they are solving a lot of the same problems. I think being at Beckman four years gave me that awareness and appreciation for other fields.”
Sullivan said there was a “very direct relationship” between what he worked on at Beckman and his current work.
“Matchmoving, image-based modeling, those are exactly the things I wo r ked on at Beckman in the AI group there,” he said.
“Then there’s the second-level stuff that is just as important,” he added, “all the classes I took at the U of I as part of the Ph.D. course work like control theory and optimization, numerical analysis. Things at the time that maybe I wasn’t so interested in but had to take as part of the degree. But they turned out to be extremely important, if only for some of the fundamental concepts like knowing if something is possible or knowing that things are related in a certain way is useful today. I don’t think I’d have that perspective had I not gone through that course and worked in that group.”
Sullivan enjoys the various rewards and honors he has received, including the Academy Award in 2001 when he received the peck from Charlize Theron.
“It was great,” he said with a laugh. “I have no reservations on that. That was really fun.”
Sullivan will be part of an ILM contingent going to the White House to be honored by President Bush with the National Medal of Technology, an award given to Arnold Beckman in 1988. The National Medal of Technology is the highest honor awarded by the President of the United States to those who have made outstanding contributions to America through technological innovation.
“Talk about the right place at the right time,” Sullivan said. “(The National Medal of Technology is) going to George and ILM for the scope of all their technology advances over the years, but it’s great to be involved in something like that.”
Ponce said he tells his classes and audiences at talks that some people may not consider computer vision useful. The he tells them about Sullivan.
“I show a picture of him with one of the Stars Wars robots and say his success has nothing to do with me. It’s his success, but the fact that this kind of work can lead to a career and impact on movies and industry is great.”
Ponce mimicked his audiences’ customary response with a reaction that would make a special effects guru proud: his eyes widen and he says, “They are like ‘ahhhhhhh.’”