Image analysis is a research field, which is rapidly growing and which can be applied in countless ways to the industry, medical science and entertainment.
Henrik Wann Jensen invented Photon Mapping at DTU, the technique that was used to simulate the skin on Gollum in "Lord of the Rings". For his research Henrik Wann Jensen received an Academy Award at the Oscar ceremony.
By Carsten Broder Hansen
The table area outside the cafe at the corner of Plaza de Mayos is one of the most popular breathing holes in Buenos Aires, for tourists and for the locals as well. It overlooks several of the most significant public buildings and – most noteworthy - the presidential palace, Casa de Gobierno, also known as "The Pink House" due to its striking color.
From my desk I had an excellent view of the palace and the famous balcony where Eva Peron spoke to the Argentinean people, and where Madonna played the same character during the making of the musical Evita. I was having a glass of red wine when suddenly the afternoon sun appeared from behind a corner and shone on my glass. Some of the rays passed through the wine, and on the backside of the glass a variety of color nuances appeared – a phenomenon known among graphic designers and image analysis as caustics.
Mathematics at the Movies
Caustics really means "burn curve" and the term is used to describe light rays that are refracted by water or glass. The effect that occurs when light passes through colored liquids in a glass and the characteristic light waves at the bottom of a swimming pool are well-known examples of caustics.
For years, caustics have been one of the core research areas in mathematic image analysis, and at the Department of Informatics and Mathematical Modeling (IMM) at DTU, calculations of the characteristics of caustics help break the boundaries of what can be done through the use of computer graphics. The mathematical models, which describe how light behaves when it hits various surfaces have been hugely useful in computer games and movies.
Caustics in Brandy
Madonna may not have won an award for her portrayal of Eva Peron, but recently the Danish researcher Henrik Wann received an Oscar for his technique for visualizing transparent materials. With great success, Henrik's technique has been used in most of the recent Hollywood blockbusters that feature animated characters.
Henrik Wann Jensen's new technique is a direct continuation of the caustics research that he did when he was a PhD student at DTU. Then, Henrik worked on photon mapping, which is a method used to calculate and simulate lighting in complex 3D set-ups, which improves the quality of the computer graphics. If using the correct method of computation, it is possible in a computer image to imitate, e.g., the complex caustics of the real world that occur when light rays hit a glass of brandy or red wine. This made it possible for Henrik, years ago, to create a life-like illusion of a glass of brandy on a sunny day.
 |
|
|
Above: A graphical emulation of the complex caustics of real life when light hits a glass of brandy. Graphics: Henrik Wann Jensen
On the right: A photograph of the phenomenon of caustics, demonstrated on a glass of red wine. Photograph: Carsten Broder Hansen
|
A "Semi-Transparent" Breakthrough
Today, Henrik Wann Jensen is a researcher in the United States, but he has maintained his connection to the Department of Informatics and Mathematical Modeling. Henrik Wann Jensen last visited IMM in January, where he gave a presentation on his latest research findings. In most graphics models, only the light that hits a surface and the reflection from the surface are usually calculated.
In the case of completely reflective surfaces such as steel, this is adequate for producing perfect images, but light's behavior in semi-transparent materials have been ignored so far, possibly because the multidimensional functions have been extremely difficult to calculate. Henrik's new technique rewrites the complex functions and turns them into more simple equations, which are able to render the same results. The new calculating method constitutes a significant breakthrough for the computer's ability to realistically represent semi-transparent materials, such as snow, marble, milk, sea water, paper and ice.
Gollum's New Skin
After the lecture, I asked Henrik Wann Jensen about the technique that had won him an Oscar. He explains: "The first computer-animated movie that featured simulated human beings was the sci-fi adventure Final Fantasy. In this movie, the computer-animated people were criticized of looking unnatural although the graphic designers had used fairly sophisticated techniques. When light hits human skin, it penetrates just a little bit and is then reflected in different ways. Particularly in places where the skin is thin, like ears and nose, a special effect occurs, which is extremely difficult to mimic in a simulation. My method now makes computer-made skin look soft, warm and more natural-looking".
The new technique's unique ability to imitate skin has gotten Hollywood excited, and it has also earned Henrik Wann Jensen a technical achievement academy award. Usually, the movie industry does not usually pay particular attention to scientific literature, so for a while, Henrik worked as a consultant at the Pixar Animation Studios to help translate mathematics into useful movie language. Dobby the elf from Harry Potter and the Prisoner of Azkaban was the first computer-generated creature, which was given a more realistic appearance with the help of the technique, and in part three of The Lord of the Rings (The Return of the King), Gollum's skin has become noticeably more natural-looking compared to what it was in The Two Towers, where the technique was not used to its full potential.
Henrik Wann does not believe that the technique is only relevant for entertainment purposes. He says: "From the point of view of universities like DTU, the debate on whether you can get children and young people interested in mathematics and physics from an early age is particularly important. For this purpose, computer graphics may prove to be a useful tool, and from a purely philosophical point of view, it is interesting that a mathematical model can be used to create a synthetic character on a computer screen".
 |
|
 |
|
Simulation of skin using traditional
techniques makes it look hard and plastic-
like in computer images.
Graphics: Henrik Wann Jensen |
|
The new technique is able to capture the
soft and natural look of the skin and
renders computer simulations that are far
more persuasive.
Graphics: Henrik Wann Jensen |
Image Analysis in the Industry
Image analysis has also influenced other different areas. Several of the sub-departments at the Department of Informatics and Mathematical Modeling collaborate closely with corporate Denmark on testing research findings as soon as they become available. Bjarne Ersbøll is Senior Lecturer at the Section for Image Analysis and Computer Graphics.
In relation to the collaboration, he explains: "In Denmark we have had great success convincing the industry to use the technology, particularly for vision systems for accurate and reliable measurements of colors and structures of different products. For instance, a patent has been taken for a measuring device, which connects a camera to a light source that emits a uniform and diffuse light. The diffuse light enhances the true colors of the product and reveals a number of patterns and color variations, which are invisible to the naked eye. Subsequently, a sophisticated camera transmits the information to the computer, which uses specialized software and digital image analysis to assess the product. These kinds of systems are used to grade mink furs, skin care products and coffee beans, etc."
Images Facilitate the Diagnostic Process
The image analysts' experience with methods and equipment used for pattern recognition has also seeped into the field of medicine. Thus, systems have been developed for decoding and comparing different skin diseases and abrasions, which makes it easier for the doctors to recognize and diagnose diseases and injuries. Also, digital images will be able to help scientists recognize and understand abnormal cardiac and vessel functions as well as malfunctions in the brain and nervous system. Our insight into how teeth and bones develop and grow has also increased significantly over the past years.
Bjarne Ersbøll explains: "For medical purposes, it may prove to be crucial that the image analysis also captures wavelengths that the eye does not see. The near-infrared and ultraviolet spectra contain loads of information, and by focusing on recording and processing this kind of information mathematically, new results can be achieved."
Visions for the Future
The inspiration for image analysis is the human vision which has been developed over the course of millions of years. We are good at making out objects and patterns in the horizon, which we probably originally developed for living on the savannah and similar surroundings. You might say that it gave us the ability to eat without being eaten. Today, scientists try to make cameras and computers emulate the characteristics of the human vision. Although, in some respect, our vision is probably close to optimal, technology offers possibilities that can be used more efficiently if we continue to research the areas and wavelengths that our vision is unable to decode.
Bjarne Ersbøll concludes: "Image analysis has become good at emulating parts of what the human vision is capable of. However, in some respects, the eye is superior, and these are the areas towards which I predict that image analysis will move in the future. One example is the ability of our vision and intelligence to cooperate on recognizing patterns. This ability is unique. When we look at a cup, we are able to decode the object in no time. Regardless of the size, shape and color, and whether the handle is turned away from us and is initially invisible, we are certain that we are looking at a cup. On the other hand, "explaining" a cup to a computer and subsequently coding the machine into recognizing the cup is an incredibly complicated process. If we are able to create a mathematical model that incorporates the many variables of a cup, we will be able to make the computer simulate the human image perception, but we are not quite there yet. How to develop a model like this is one of the great challenges of image analysis."