CVR Presents New Techniques at Siggraph 2005

A paper presentation at Siggraph 2005 in Los Angeles last month by Hongcheng Wang, a member of the Computer Vision and Robotics Laboratory, enlightened thousands attending the annual graphics convention about techniques the group developed for compactly representing multidimensional visual datasets for efficient image-based rendering on a PC.

Dynamic Bidirectiona Texture Functionsl

The presentation was based on a paper by Professor Narendra Ahuja (AI) and his student Wang, in collaboration with Professor Yizhou Yu and his students Qing Wu and Lin Shi, called Out-of-Core Tensor Approximation of Multi-Dimensional Matrices of Visual Data. Included in the presentation was the first-ever 7-D dynamic Bidirectional Texture Function (BTF) video, produced by the lab.

The paper describes their techniques for compactly representing multidimensional visual datasets for efficient image-based rendering on a PC. Operating directly on the compact representation, they were able to efficiently interpolate new images that included the dimensional information that often makes scenes so compelling - and that usually gets lost in image transfer methods.

Using their methods, the group began with a set of images of a wave-filled swimming pool from a sparse number of viewing directions and under a limited variety of lighting conditions. They then showed what the same scene would look like at different times of the day as the light changes, and from continuously varying perspectives, and with the waves moving faster or the water becoming placid. The continuous, interpolated images introduced temporal, spatial and illumination changes in the scene - factors missing in the original image set - in close to real time.

"What we have done gives us the ability to redisplay a scene, i.e., to create a video, which depicts a new viewpoint, or new lighting conditions, or new display speeds, starting with a relatively small number of images of the original scene," Ahuja said.

Ahuja said that one reason for this success is that, unlike many image processing algorithms that flatten data into a linear string of numbers, thus losing the multidimensional information, their representations retain the original, higher dimensionality of the space.

"When we do this approximation to discover the structure, which in turn enables us to do the animation, we don't lose the original dimensionality of the data, which is very important," Ahuja said.

"In addition to (that) capability, an independent, desirable feature of this work is efficient utilization of storage. Normally, this kind of image processing would require huge amounts of memory, or core, which is usually prohibitive for a personal computer.

"We had to devise an algorithm which had to work in the normally available amounts of computer memory and be able to operate on a part of the data at a time, and yet perform the same overall computations over time. So you're not paying any price in quality for having less memory; the result is still the same as if you had infinite memory."

Ahuja, who is director of the Computer Vision and Robotics Laboratory, and a member of the Artificial Intelligence group, said applications could include uses in animation and movies.