1、Introduce
A stereo 3D camera is a type of camera equipped with two or more lenses, each with its own image sensor or film frame. This configuration allows the camera to capture images from slightly different angles, simulating the human binocular vision. The primary purpose of a stereo 3D camera is to provide depth perception and create 3D images. The Gemini 330 series 3D cameras combine active and passive stereo vision technologies for seamless operation in both indoor and outdoor conditions.
Working Principle:
- Stereoscopic Vision: By capturing two images from different perspectives, a stereo 3D camera can calculate the distance to objects in the scene. This process involves measuring the disparity between the two images and using it to triangulate the depth.
- Image Processing: Advanced algorithms are used to analyze the images captured by the two lenses. These algorithms match corresponding points in both images to compute depth information, generating a 3D representation of the scene.
For the Orbbec Gemini 330 series cameras, the implementation of the working principle is completed on the Orbbec’s latest custom ASIC.
Figure 1. Gemini 335 Stereo 3D Camera
2、Baseline
The baseline is the distance between the two cameras in a stereo setup, namely, the distance between two IR cameras as shown in Figure 1. A longer baseline increases depth accuracy but also increases the size of the setup. Proper baseline selection is crucial for balancing depth resolution and device compactness.
3、Field of View(FOV)
The FOV of a 3D camera determines the angular extent of the observable world that can be captured. A larger FOV allows the camera to capture more area but may reduce the resolution.
For the depth is calculated from the left and right images, the FOV of the depth camera is determined by the FOV of the left or right cameras, the measured distance and the baseline. The closer the distance, the smaller the FOV; the further the distance, the closer the FOV of the depth camera is to the FOV of the left or right camera.
Figure 2. Depth FOV and IR FOV
Depth Field of View (Depth FOV) at any depth (Z) can be calculated using the following equation:
Definitions:
- cx: X-direction image coordinates of the principle point of the depth image.
- fx: Depth camera focal length.
- width: Depth image width.
Figure 3. Gemini 335 Depth FOV for Aspect Ratio = 16:10
Figure 4. Gemini 335 Depth FOV for Aspect Ratio = 16:9
4、Blind Spots
Blind spots in 3D cameras are areas where the camera cannot capture depth information accurately. These are usually found at the edges of the FOV or the closest distance between the camera and the object. When the object is not visible in either the left or right image, accurate depth information cannot be obtained. The depth camera uses the left camera as the main camera, and the depth blind spots are the green and gray areas shown in the figure below.
Figure 5. Depth blind spots
5、Depth Dark Borders
Depth black edges, or dark borders, in stereo 3D cameras refer to areas in the depth map where depth information is either missing or incorrect. These edges often appear at the boundaries of objects where there is a significant depth discontinuity.
Cause of Depth Dark Borders:
- Baseline: Stereo 3D cameras work by capturing two images from slightly different perspectives and then calculating the disparity (difference) between these images to estimate depth. These occur when objects seen in one image are not visible in the another due to FOV and baseline limitations, as shown in the green blind spots above.
- Occlusions: These occur when parts of one image are not visible in the other due to obstruction by an object. This leads to missing depth information, often resulting in black edges in the depth map.
Figure 6. Depth Dark Borders
6 、Depth Error
The errors of stereo 3D cameras are primarily influenced by disparity errors, which are related to the reflectivity of the measured object, image signal-to-noise ratio, algorithm errors, and other factors.
Depth is consequently derived from this equation:
By differentiating the disparity, we can derive the depth error as follows:
It can be seen that the depth error of binocular vision increases parabolically (proportional to the square of the distance) as the distance increases.
Figure 7. Depth error(mm) as the distance(mm) increases
