Mosaic (video panorama) generation is the process of generating the background from a video captured through a moving camera. Our work was supported by NSF. Prior to our research, majority of the research targeted a small set of videos for mosaic generation research. We have categorized videos for mosaic generation and proposed that methods should not be video specific. We have developed the "Sprite Fusion" technique for tracking videos without foreground/background segmentation while removing the objects. Another significance of our work is that the sprite can be generated even 90% of the scene is covered. We have developed the virtual camera system called "SpriteCam" by providing virtual camera controls using the sprite. We have provided examples of aspect ratio conversion without cropping or lowering resolution.

NSF Project Description

This project provides solutions to two main important components of an interactive, object-based, semantic multimedia information retrieval system: mosaic generation and efficient indexing and retrieval of spatio-temporal content of the video. A Mosaic can be considered as a static component (or background) of a scene that does not change over a sequence of frames and is obtained by computing the global motion between frames, warping according to the global motion, and then blending the frames. Mosaic generation plays an important role in many applications including object-based coding (where objects in a scene are also coded or compressed independent of regular rectangular frame coding), video compression, video indexing, object tracking, virtual environments, security surveillance, wide-area surveillance, panoramic video, traffic monitoring, object recognition, and human behavior analysis since these applications usually require the subtraction of actual scenes from the background (or the mosaic) to determine the foreground objects. These bring the challenges of the mosaic generation: limited domain of videos for mosaic generation, accuracy and reliability. The first section of this project can be divided into two subsections. The first section (i) develops mosaic generation solutions for larger domains of videos according to considering suitability to generate sprite from the video and objects; (ii) imports a novel blending algorithm for a specific set of tracking videos including real videos and synthetic videos with mobile or static object (objects). The second subsection presents: (i) mosaic generation solutions for larger domains of videos; (ii) mosaics for videos containing many shots by classifying video shots; and (iii) objective evaluation methods we developed for mosaic generation by producing ground-truths. We also work on interactive video reproduction from the available videos in two ways: using mosaic and using available video database. We use the generated mosaic and overlay the objects to interactively generate the videos. In addition, we index the videos and after analyzing the similarities, we use the video database for video reproduction. This part also includes interaction video reproduction using mosaics and extending part to applying mosaic generation and visualization for high- definition video.

Our research is maintained at our main video server at Please visit our updated website.

This material is based upon work supported by the National Science Foundation under Grant No. 0812307. 


  1. Y. Chen, R.S. Aygun. “SpriteCam: virtual camera control using sprite,” Multimedia Tools and Applications, October 2013. DOI: 10.1007/s11042-013-1711-6
  2. Yi Chen, Abhidnya A. Deshpande, and Ramazan S. Aygüun. 2012. Sprite generation using sprite fusion. ACM Trans. Multimedia Comput. Commun. Appl. 8, 2, Article 22 (May 2012), 24 pages.
  3. M. Naik, M. Sigdel, and R. Aygun. “Spatio-Temporal Querying Recurrent Multimedia Databases Using a Semantic Sequence State Graph Multimedia Systems,” Multimedia Systems Journal, Vol. 18, Issue 3, pp. 263-281, June 2012
  4. Y Chen and Ramazan S. Aygun. “Synthetic Video Generation for Evaluation of Sprite Generation,”2010 International Journal of Multimedia Data Engineering and Management (IJMDEM), Vol. 1, No. 2, pp. 34-61, 2010
  5. Dinc, S., Fahimi, F., Aygun, R.S., “GPU Based Robust Image Registration for Composite Translational, Rotational and Scale Transformations,” 2015 IEEE International Symposium on Multimedia (short paper, accepted)
  6. S. Dinc, M. Sigdel, I. Dinc, MS. Sigdel, F. Fahimi, R. Aygun. “Depth-Color Image Registration for 3D Surface Texture Construction using Kinect Camera System,” IEEE SoutheastCon 2014, Lexington, KY, March 13-16, 2014
  7. Yi Chen, Ramazan Savas Aygün: Improving Global Motion Estimation Using Texture Masks. ISM 2011: 583-588
  8. Vineetha Bettaiah and Ramazan Aygun, “A User Interface for Spatio-Temporal ‘Eventually’ Queries using Gamepad,” Advances in Multimedia, 2011. MMEDIA '11. Third International Conference on , MMEDIA 2012, pp. 38-43, Budapest, Hungary, 17 April 2011
  9. Yi Chen; Aygun, R.S.; , "Synthetic Video Generation with Camera Motion Patterns to Evaluate Sprite Generation," Advances in Multimedia, 2009. MMEDIA '09. First International Conference on , vol., no., pp.140-145, 20-25 July 2009
  10. Deshpande, A.A.; Aygun, R.S.; , "Motion Based Video Classification for SPRITE Generation," Database and Expert Systems Application, 2009. DEXA '09. 20th International Workshop on , vol., no., pp.231-235, Aug. 31 2009-Sept. 4 2009
  11. Yi Chen; Aygun, R.S.; , "Synthetic Video Generation with Complex Camera Motion Patterns to Evaluate Sprite Generation," Multimedia, 2009. ISM '09. 11th IEEE International Symposium on , vol., no., pp.657-662, 14-16 Dec. 2009
  12. Yi Chen; Deshpande, A.A.; Aygun, R.S.; , "SpriteDB Tool A Web-Tool for Comparing Sprite Generation Results," Multimedia, 2009. ISM '09. 11th IEEE International Symposium on , vol., no., pp.438-439, 14-16 Dec. 2009
  13. Ramazan Aygun, Yi Chen, Vineetha Bettaiah, Thejaswi Raya, and Ayesha Bhatnagar, “Mosaic Generation: Challenges & Future Directions,” NSF III 2010 Workshop, April 22-23, 2010 (abstract-poster only)
  14. Ramazan Aygun, “Object Tracking and Solving Image Puzzles using CmuCam2+ Vision Sensor,” University of Alabama in Huntsville Young Faculty Research Proceedings, 2007.
  15. Ramazan Savas Aygun and Aidong Zhang. "Extracting Coarse Boundary Features for Video Processing", 2002 IEEE International Conference on Multimedia and Expo, Lausanne, Switzerland, August, 2002, Volume 2, pp. 65-68
  16. Ramazan Savas Aygun and Aidong Zhang. "Reducing Blurring-Effect in High Resolution Mosaic Generation", 2002 IEEE International Conference on Multimedia and Expo, Lausanne, Switzerland, August, 2002, Volume 2, pp. 537-540
  17. Ramazan Savas Aygun and Aidong Zhang. "Global Motion Estimation from Semi-Dynamic Video using Motion Sensors", 2002 IEEE International Conference on Image Processing, Rochester, New York, September, 2002, Volume 2, pp. 273-276
  18. Ramazan Savas Aygun and Aidong Zhang. “Sprite Pyramid for Videos and Images Having Finite-Depth Scenes”, 2004 IEEE Conference on Multimedia and Expo, Taipei, Taiwan, June, 2004.
  19. Ramazan Savas Aygun and Aidong Zhang.. “Integrating Virtual Camera Controls into Digital Video”, 2004 IEEE Conference on Multimedia and Expo, Taipei, Taiwan, June, 2004
  20. Ramazan S. Aygun and Aidong Zhang. "Stationary background generation from MPEG compressed video sequences", 2001 IEEE International Conference on Multimedia and Expo, Tokyo, August, 2001, pp. 908-911