A Framework for Fusion of 3D Appearance and 2D Shape Cues for Generic Object Recognition

This paper addresses the problem of Generic Object Recognition (GOR) from arbitrary viewpoints by modeling the perceptual capability of human beings. We propose a novel framework which uses a combination of 3D appearance and 2D shape cues to recognize the object class as well as determine its pose. We propose a hierarchical framework for GOR, which combines two stages of processing. First, the 3D appearance model of the object is captured from multiple viewpoints using Linear Subspace Analysis techniques. These appearance cues are used to reduce the search space to a few rank-ordered samples. We have used a decision-fusion based combination of 2D PCA and ICA to integrate the complementary information of classifiers and improve appearance-based recognition accuracy. Shape matching is then performed on the reduced search space, using either distance transform based correlation or shape context based matching. The proposed framework for GOR uses a decision fusion technique, in which evidences from 3D appearance and 2D shape are combined (fused) to obtain the correct object class and its pose. Experiments were conducted using objects with complex appearance and shape characteristics, and the performance of the proposed framework has been shown to be superior, using the COIL-100 and IGOIL (IITM Generic Object Image Library) databases. IGOIL database was also used to analyze the appearance manifolds along two orthogonal axis of rotation. Performance degradation in case of noisy images has also been presented.