Stereo disparity cores
The depth estimation can be based on the correspondences of image features of a binocular system. Disparity computation models basically consist of a matching problem, finding correspondences between features or areas in both left and right images and taking into account some constraints based on the geometry of the stereo rig that may help to simplify the search process (due to the epipolar geometry constraint). This model assumes that the geometry of the cameras is fixed and known (e.g., using a previous calibration stage that allows us to extract the intrinsic and extrinsic binocular cameras parameters). Based on this assumption, in most of the literature contributions, the disparity is managed as a mono-dimensional matching problem of correspondences.
Different estimation methods usually assume that images are calibrated or include a pre-processing stage for undistortion and rectification. This process consists of correcting the lens radial distortions and aligning the image planes with the epipolar lines, which simplifies the disparity estimation, reducing the problem to a matching only for the horizontal coordinates. Nevertheless, the main problem is that the rectification process is required each time the camera configuration is modified.
Depth perception is essential for an autonomous system that is moving in a dynamic environment. It is applied in multiple applications such as autonomous navigation, obstacle detection and avoidance, 3-D reconstruction, tracking, grasping.
Our cores are able to process 640 by 480 images at a frame rate up to 32 fps for the Lucas Kanade algorithm including undistortion and rectification stage (see Fig 1 for some examples). Calibration matrices have to be provided and previously calculated offline. Cores are implemented in Handel-C and detailed in [2][3][5].
Fig 1. Disparity estimation for some examples from the Middlebury
datasets.
As mentioned in Image-processing-core-library, all the work is supported by several publications in international scientific journals. However, the main difference is that here we are only including the cores for the one-scale computation, not for the whole coarse-to-fine process. The reason is that the coarse-to-fine structure requires intensive use of the memory and therefore, it is platform-dependent in our case.
Gradient-based model
An implementation of the Lukas and Kanade adaptation from Bouguet [1] is provided in this section. The problem of estimating disparity is reduced to a one-dimensional problem. The hardware architecture for the disparity estimation core is shown in Fig 2. Assuming the intensity constancy and the disparity to change smoothly, this method estimates efficiently the depth of the objects in the scene. More details on the hardware implementation can be found in [2][3] and in img-proc-core-lib:Papers.
Fig 2. Scheme of the architecture for the disparity estimation core.
Phase-based model
Implementation based on Solari and Sabatini [4].
Contacts
- Javier Diaz Seven Solutions
- Matteo Tomasi Harvard University
- Francisco Barranco University of Maryland and University of Granada
Project Status
Date | Release |
---|---|
10/02/2014 | v1.0 Release |
References
[1] J. Y. Bouguet. Pyramidal Implementation of the Lucas Kanade
Feature Tracker
[2] F. Barranco, M. Tomasi, J. Díaz, M. Vanegas, E. Ros, “Pipelined
architecture for real-time cost-optimized extraction of visual
primitives based on FPGAs”, Digital Signal Processing, vol. 23, n. 2,
pp. 675-688, 2013.
[3] F. Barranco, J. Díaz, A. Gibaldi, S.P. Sabatini, E. Ros, “Vector
disparity sensor with vergence control for active vision systems”,
Sensors, vol. 12, pp. 1771-1799, 2012.
[4] F. Solari, S.P. Sabatini, G. M. Bisio: Fast technique for
phase-based disparity estimation with no explicit calculation of phase,
Electronics Letters, 37 (23): 1382-1383 (2001)
[5] M. Tomasi, M. Vanegas, F. Barranco, J. Diaz, E. Ros: Real-Time
Architecture for a Robust Multi-Scale Stereo Engine on FPGA. IEEE Trans.
VLSI Syst. 20(12): 2208-2219 (2012)