drag and drop web creator


Interactive Spatio-Temporal Querying Using a Gaming Controller

One of the biggest challenges of spatio-temporal querying is the difficulty of setting up user queries. Since users need to provide both spatial and temporal information for different time instants, getting such information is not simple.

Firstly, we need a proper index structure for fast retrieval of data from the database. We have developed Semantic Sequence State Graph (S3G) for spatio-temporal indexing. The spatial context is limited. Objects move their positions on spatial context and these movements might be results of some actions. S3G can effectively index such databases. We have applied S3G on tennis video databases.

There are two aspects of Query-by-Gaming: 1) finding relevant clips and showing those clips to the user as inputs from a gaming controller is taken, and 2) finding clips that meet the criteria of of the user. The queries can be provided in a sequential mode or eventual mode. These modes are based on linear temporal logic. For sequential mode, 'next' operator is used. For the eventual mode, 'eventually' operator is used.


  1. V. Bettaiah, R. Aygun. “Query-by-Gaming: Interactive Spatio-Temporal Querying and Retrieval Using Gaming Controller,” Journal of Visual Languages and Computing, Vol 29, August 2015, pp. 63-76. doi:10.1016/j.jvlc.2015.03.003
  2. M. Naik, M. Sigdel, and R. Aygun. “Spatio-Temporal Querying Recurrent Multimedia Databases Using a Semantic Sequence State Graph Multimedia Systems,” Multimedia Systems Journal, Vol. 18, Issue 3, pp. 263-281, June 2012
  3. Vineetha Bettaiah and Ramazan Aygun, “A User Interface for Spatio-Temporal ‘Eventually’ Queries using Gamepad,” Advances in Multimedia, 2011. MMEDIA '11. Third International Conference on , MMEDIA 2012, pp. 38-43, Budapest, Hungary, 17 April 2011
  4. Mitesh Naik, Vani Jain, and Ramazan Aygun. “S3G: A Semantic Sequence State Graph for Indexing Spatio-Temporal Data - a Tennis Video Database Application,” 2008 IEEE Int. Conference on Semantic Computing, pp. 66-73, Santa Clara, CA, August, 2008