Multispectral Video Fusion

Fusion Result

Department of Computer Science The University of North Carolina at Chapel Hill Chapel Hill, NC

Abstract

Input RGB Input IR

We present a video fusion technique for enhancing underexposed visual spectrum footage by using simultaneously captured video from non-visible spectrum sensors, such as short wave IR or near IR. Although IR sensors can accurately capture video in low-light and night-vision applications, these sensors lack the color and the relative luminances of visible spectrum sensors. RGB sensors capture color and correct relative luminances, but are underexposed, noisy, and lack fine features. With a temporal noise reduction preprocess, we remove shot noise and increase color accuracy in static areas of the RGB footage. The IR video is then normalized to ensure cross-spectral compatibility with the RGB video. A ratio image determined by a mutual information version of the bilateral filter is used to control the normalization. Finally, the RGB low frequencies and IR details are fused together along with all associated high-frequency edges to create a noise reduced video with sharp details, proper chrominances, and natural relative luminances.

People

Eric P. Bennett & John L. Mason & Leonard McMillan

Publications

Eric P. Bennett, John L. Mason, and Leonard McMillan "Multispectral Video Fusion" SIGGRAPH 2006 (Boston, MA)
SIGGRAPH Pre-Print PDF (.4 MB)

Video

SIGGRAPH Supplemental Video (MPEG-4, 26.8 MB)
This video is encoded with MPEG-4 in Apple QuickTime, available here.

Contact

If you are interested in further details, please contact Eric Bennett