As early stage of video processing, we introduce an iter- ative trajectory merging algorithm that produces a region- based and hierarchical representation of the video se- quence, called the Trajectory Binary Partition Tree (BPT). From this representation, many analysis and graph cut tech- niques can be used to extract partitions or objects that are useful in the context of specific applications. In order to define trajectories and to create a precise merging algorithm, color and motion cues have to be used. Both types of informations are very useful to characterize objects but present strong differences of behavior in the spa- tial and the temporal dimensions. On the one hand, scenes and objects are rich in their spatial color distributions, but these distributions are rather stable over time. Object mo- tion, on the other hand, presents simple structures and low spatial variability but may change from frame to frame. The proposed algorithm takes into account this key difference and relies on different models and associated metrics to deal with color and motion information. We show that the proposed algorithm outperforms existing hierarchical video segmentation algorithms and provides more stable and pre- cise regions

Peer Reviewed