Content-based inter-media synchronization

Inter-media synchronization methods developed until now have been based on syntactic timestamping of video frames and audio samples. These methods are not fully appropriate for the synchronization of multimedia objects which may have to be accessed individually by their contents, e.g. content-based data retrieval. We propose a content-based multimedia synchronization scheme in which a media stream is viewed as hierarchical composition of smaller objects which are logically structured based on their contents, and the synchronization is achieved by deriving temporal relations among logical units of media object. Content-based synchronization offers several advantages such as, elimination of the need for time stamping, freedom from limitations of jitter, synchronization of independently captured media objects in video editing, and compensation for inherent asynchronies in capture times of video and audio.