Mosaic (video panorama) generation is the process of generating the background from a video captured through a moving camera. Our work was supported by NSF. Prior to our research, majority of the research targeted a small set of videos for mosaic generation research. We have categorized videos for mosaic generation and proposed that methods should not be video specific. We have developed the "Sprite Fusion" technique for tracking videos without foreground/background segmentation while removing the objects. Another significance of our work is that the sprite can be generated even 90% of the scene is covered. We have developed the virtual camera system called "SpriteCam" by providing virtual camera controls using the sprite. We have provided examples of aspect ratio conversion without cropping or lowering resolution.
This project provides solutions to two main important components of an interactive, object-based, semantic multimedia information retrieval system: mosaic generation and efficient indexing and retrieval of spatio-temporal content of the video. A Mosaic can be considered as a static component (or background) of a scene that does not change over a sequence of frames and is obtained by computing the global motion between frames, warping according to the global motion, and then blending the frames. Mosaic generation plays an important role in many applications including object-based coding (where objects in a scene are also coded or compressed independent of regular rectangular frame coding), video compression, video indexing, object tracking, virtual environments, security surveillance, wide-area surveillance, panoramic video, traffic monitoring, object recognition, and human behavior analysis since these applications usually require the subtraction of actual scenes from the background (or the mosaic) to determine the foreground objects. These bring the challenges of the mosaic generation: limited domain of videos for mosaic generation, accuracy and reliability. The first section of this project can be divided into two subsections. The first section (i) develops mosaic generation solutions for larger domains of videos according to considering suitability to generate sprite from the video and objects; (ii) imports a novel blending algorithm for a specific set of tracking videos including real videos and synthetic videos with mobile or static object (objects). The second subsection presents: (i) mosaic generation solutions for larger domains of videos; (ii) mosaics for videos containing many shots by classifying video shots; and (iii) objective evaluation methods we developed for mosaic generation by producing ground-truths. We also work on interactive video reproduction from the available videos in two ways: using mosaic and using available video database. We use the generated mosaic and overlay the objects to interactively generate the videos. In addition, we index the videos and after analyzing the similarities, we use the video database for video reproduction. This part also includes interaction video reproduction using mosaics and extending part to applying mosaic generation and visualization for high- definition video.
Our research is maintained at our main video server at http://sprite.cs.uah.edu/mosaics. Please visit our updated website.
This material is based upon work supported by the National Science Foundation under Grant No. 0812307.
