On disparity matching in stereo vision via a neural network framework

Abstract

This paper presents a neural framework for dealing with the problem of disparity matching in stereo vision. Two different types of neural networks are used in this framework: one is called the vitality conservation (VC) network for learning clustering, and the other is the back-propagation (BP) network for learning disparity matching. The VC network utilizes a vitality conservation principle to facilitate self-development in network growing. The training process of VC is smooth and incremental; it not only achieves the biologically plausible learning property, but also facilitates systematic derivations for training parameters. Using the [intensity, variation, orientation, x, y] of each pixel (or a block) as the training vector, the VC network dismembers the input image into several clusters, and results can be used by the BP network to achieve accurate matching. Unlike the conventional k-means and self-organizing feature map (SOFM), VC is a self-creating network; the number of clusters is self-organizing and need not be pre-specified. The BP network, using differential features as input training data, can learn the functional relationship between differential features and the matching degree. After training, the BP network is first used to generate an initial disparity (range) map. With the clustering results and the initial map, a matching algorithm that incorporates the BP network is then applied to recursively refine the map in a cluster-bycluster manner. In the matching process, useful constraints, such as a epipolar line, ordering, geometry and continuity, are employed to reduce the occurrence of mismatching. The matching process continues until all clusters are matched. Empirical results indicate that the proposed framework is very promising for applications in stereo vision.