A Principal Component Clustering Approach to Object-Oriented Motion Segmentation and Estimation

1. Abstract:
   This paper presents a framework for object-oriented scene segmentation in video, which uses motion as the major characteristic to distinguish different moving objects and then to segment the scene into object regions. From the feature block (FB) correspondences through at least two frames obtained via a tracking algorithm, the reference feature measurement matrix and feature displacement matrix are formed. We propose a technique for initial motion clustering of the FBs, in which the principal components (PC) of the two matrices are adopted as the motion features. The motion features have several advantages: (1) They are low-dimensional (2-dim). (2) They preserve well both the spatial closeness and the motion similarity of their corresponding FBs. (3) They tend to form distinctive clusters in the feature space, thus allowing simple clustering schemes to be applied. The Expectation-Maximization (EM) algorithm is applied for clustering the motion features. For those scenes involving mainly camera motion, the PC-based motion features will exhibit nearly parallel lines in the feature space. This facilitates a simple and yet effective layer extraction scheme. The final motion-based segmentation involves labeling of all the blocks in the frame. The EM algorithm is again applied to minimize an energy function which takes motion consistency and neighborhood-sensitivity into account. The proposed algorithm has been applied to several test sequences and the simulation results suggest a promising potential for video applications.
2. Remarks:
   
   1. The research was conducted in collaboration with Yun-Ting Lin, under the direction of Professor S. Y. Kung.
   2. The paper on this research is presented in the Journal of VLSI Signal Processing Systems for Signal, Image, and Video Technology, vol. 17, no. 2, pp. 163--188, Nov. 1997.