Yong Rui knows how many mobile phone users there are in China (nearly twice the U.S. population) and when the Chinese will overtake Americans as the most frequent users of the Internet (next year).
As technical assistant to Microsoft's top boss in the company's China R&D Group, Rui knows where China has been and where it is in terms of technology development. Since January of this year, Rui now has a say in where some of that development is headed.
Rui grew up in China, graduated from its top engineering college, and studied computer vision at the Beckman Institute from one of the field's pioneers, Human- Computer Intelligent Interaction Co-chair Thomas Huang. After graduating from the University of Illinois with a Ph.D. in electrical and computer engineering in 1998, Rui joined Microsoft Research and subsequently became head of the Multimedia Collaboration team at the company's headquarters in Redmond, Washington.
During those same years, Microsoft was building its presence in China. After starting a sales office and a research and development center there in the 1990s, the software giant launched Microsoft Research China, later renamed Microsoft Research Asia, in 1998. Other research and technology development centers followed until the company decided to establish an umbrella organization in 2006 called the Microsoft China Research and Development Group. Microsoft chose Corporate Vice-President Ya-Qin Zhang to be president of this new group, with Rui serving as Zhang's top technical adviser.
The group's purpose, says Rui, is to guide Microsoft's research and development directions in China. The group will explore new research lines for Microsoft applications while also developing products geared toward China's burgeoning consumer base.
China's stunningly rapid emergence as a major player on the world's economic stage is having a huge impact on everything from manufacturing to technology, and Microsoft plans on being a part of it. Rui's responsibility, he said, is “to help the president define and drive the overall R&D strategy in China.”
That's a big task, but one for which Rui is uniquely qualified. The knowledge Rui gained at Beckman and UIUC in computer vision, signal processing, and machine learning, he applied to developing communication and multimedia systems at Microsoft. Over the years he has published research papers and book chapters with Huang and many others, remaining in the forefront of image and signal processing and multimedia issues.
Rui's focus for his Ph.D was on image retrieval, and it turned out to be the first of several smart choices. Rui found himself in a perfect position while working toward his doctorate when UIUC was selected as one of five universities to take part in a seminal government-sponsored digital library project.
“I was very lucky,” he said. “I was Tom's first student who was doing research in this particular area, because it was very, very new. A new field can be both good and bad — good in the sense that if you discover something, then wow, you can really make a big impact. The not so good part is you don't have much to learn from because it is so new. I was lucky in the sense that I actually discovered something in the field.”
Rui's contribution was relevant feedback, which uses algorithms for image retrieval that work like a text search on Internet search engines. It dovetailed nicely with the digital library project.
“I was lucky in that I entered this field at the right time and I introduced this technology,” Rui said. “Now it's pretty much everywhere. This by itself is already a research direction in the multimedia research community. So I'm very happy and proud that I did something in that field.”
Rui's talent undoubtedly played a bigger role than luck, but he does seem to be in the right place at the right time. While the move to China may seem like an unbridled opportunity to affect important research and development over the next few years, it was also a big change for Rui. He left his research group behind, and moved his wife and four-year-old to Beijing in March.
“Sometimes you just have to take risks,” Rui said. “For example, my move to China in some sense is risky because I'm doing pretty well here in our headquarters. Why would I want to take a risk and go to China and spend a few years there? Sometimes you need to leave your previous success behind and move on to the next big challenge with a peaceful mindset.”
Rui believes Microsoft China R&D will have a big impact because of the research and development areas it is focusing on and because China is so ripe with opportunity.
“There are three key areas that Microsoft China R&D Group will be focusing on: mobile and embedded systems, digital entertainment, and Internet technologies and services. I think these areas are very carefully chosen so that it makes sense to be in China,” Rui said. “China is already the No. 1 market in mobile telephones. Today it has over 430 million users. It's also No. 1 in consumer electronics as a manufacturer and as consumers. That's why digital entertainment is very big there.”
Rui said online gaming is also a booming market in China and other Asian countries, and the number of Internet users there is growing exponentially.
“Today, China is No. 2, but next year it will likely pass the U.S in the number of Internet users, people who have Internet access,” he said. “All three areas are very exciting. You're looking at a huge market. You want to do innovations, bring in new research technology and transfer that into products and bring it to the market. So that by itself is very, very exciting.”
Rui's focus in his new position will be different from the research he did with his old team. As part of Microsoft's Communication and Collaboration systems group, the team he managed worked on creating innovative multimedia collaboration systems for meetings and seminars. They developed a system called Automated Lecture Rooms, or iCam, that features localized sound detection microphones and realtime person-tracking cameras to record meetings and lectures automatically in the same manner as a live camera crew. Rui and his colleagues also developed a Ring- Cam recording system featuring an array of cameras at the center of a table, and Worklounge for integrating videoconferencing tools.
The iCam has been used for six years at Microsoft Research's headquarters, recording more than 600 lectures seen by more than 20,000 visitors. Rui has developed two technologies that have shipped commercially as part of Microsoft products. Microsoft's Moviemaker 1.0. for digitalizing home video includes a frame index developed by Rui. It uses a shot boundary locator based on when the camera is turned on and off to find the frames users are looking for, using a key frame to represent a shot sequence. Another is a bandwidth estimator for collaborators communicating via the Internet. It tells end point users how much bandwidth they have available for sharing files.
Although his work at Microsoft has differed somewhat from his research at Beckman, Rui said he was prepared well at UIUC for his future endeavors. He credits Huang for providing both freedom and opportunity for his students.
“He gave some high-level directions and suggestions,” Rui said. “I would try something and tell Tom 'oh this works, this doesn't work. Do I try A or B or C?' Tom has a lot of experience and would say 'oh C won't work because I tried that many years ago.' Also Tom has this very open approach. He gave me a lot of freedom so I could try many different things.”
That freedom led to his discovery of relevant feedback in image retrieval, Rui said.
“Relevant feedback was not in this image processing or computer vision field,” he said. “I actually read many unrelated things. I borrowed this idea from text-based retrieval. It had nothing to do with my image video analysis. Because of Tom's openness I was able to explore many other fields and discovered this was a good algorithm to be used in image and video analysis.”
The interdisciplinary aspect of working at Beckman was also helpful.
“When I talked to other people who are doing bioengineering or something involving people from very different fields, the discussions sometimes led me to look at problems from different perspectives,” Rui said.
Recently, Rui and Microsoft China R&D Group President Zhang toured 10 of the top universities in China. Rui offered the students advice based on the acronym SCORE. Rui said it starts with 'S for having a solid foundation in whatever field a student is studying.
“You have to understand your particular field in a very solid way — in double E you have to know signal processing very well, in computer science you better know how to write a compiler or how an operating system is working,” he said.
But technical knowledge only will take a student so far, Rui said. Communication is just as important.
“Many times what helps you to be successful in your field after college is not only solid knowledge, the hard skills, but also the soft skills.” Rui said. “How do you communicate, how do you make people understand you and you listen to them. So the C skill is very important.”
Rui also advises keeping an open mind (O) and taking risks (R), which he has done by moving to China. Lastly, he counsels students to be enterprising (E).
“We encourage people to think not only on the technical side but they should also think about it from an end-user's point of view,” Rui said. “There are different models. Sometimes you start with a product and look for a scenario. Other times you start with a scenario and you find the right technology. For technology to be successful you should have good knowledge but also have an entrepreneurial mindset. Hopefully if a student follows this they will score well.”
Rui followed his own advice and now is in a position to have a real effect on the future of technology in his native country and the world.