Spatio-temporal descriptor using 3D curvature scale space

This paper presents a novel technique to jointly represent the shape and motion of video objects for the purpose of content based video retrieval (CBVR). It enables to retrieve similar objects undergoing similar motion patterns, that are not captured only using motion trajectory or shape descriptors. In our approach, both shape and motion information are integrated in a unified spatio-temporal representation. Curvature scale space theory proposed by Mokhtarian is extended (in 3D) to represent shape as well as motion trajectory of video objects. A sequence of 2D contours are taken as input and convolved with a 2D Gaussian. The zero crossings are found out from the curvature of evolved surfaces, which form the 3D CSS surface. The peaks from the 3D CSS surface form the features for joint spatio-temporal representation of video objects. Experiments are carried out on CBVR and results show good performance of the algorithm. © Springer-Verlag Berlin Heidelberg 2007.