Enhanced RGB-D Mapping Method for Detailed 3D Indoor and Outdoor Modeling.
RGB-D sensors (sensors with RGB camera and Depth camera) are novel sensing systems that capture RGB images along with pixel-wise depth information. Although they are widely used in various applications, RGB-D sensors have significant drawbacks including limited measurement ranges (e.g., within 3 m) and errors in depth measurement increase with distance from the sensor with respect to 3D dense mapping. In this paper, we present a novel approach to geometrically integrate the depth scene and RGB scene to enlarge the measurement distance of RGB-D sensors and enrich the details of model generated from depth images. First, precise calibration for RGB-D Sensors is introduced. In addition to the calibration of internal and external parameters for both, IR camera and RGB camera, the relative pose between RGB camera and IR camera is also calibrated. Second, to ensure poses accuracy of RGB images, a refined false features matches rejection method is introduced by combining the depth information and initial camera poses between frames of the RGB-D sensor. Then, a global optimization model is used to improve the accuracy of the camera pose, decreasing the inconsistencies between the depth frames in advance. In order to eliminate the geometric inconsistencies between RGB scene and depth scene, the scale ambiguity problem encountered during the pose estimation with RGB image sequences can be resolved by integrating the depth and visual information and a robust rigid-transformation recovery method is developed to register RGB scene to depth scene. The benefit of the proposed joint optimization method is firstly evaluated with the publicly available benchmark datasets collected with Kinect. Then, the proposed method is examined by tests with two sets of datasets collected in both outside and inside environments. The experimental results demonstrate the feasibility and robustness of the proposed method.
State Key Laboratory of Information Engineering in Surveying Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, China. Shengjun.tang@whu.edu.cn.|State-Province Joint Engineering Laboratory of Spatial Information Technology for High Speed Railway Safety, Chengdu 610031, China. Shengjun.tang@whu.edu.cn.|Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 610031, China. Shengjun.tang@whu.edu.cn.|Collaborative Innovation Center for Geospatial Techneology, 129 Luoyu Road, Wuhan 430079, China. Shengjun.tang@whu.edu.cn.|Department of Land Surveying & Geo-Informatics, The Hong Kong Polytechnic University, Hung Hom 999077, Hong Kong, China. Shengjun.tang@whu.edu.cn.
Shengjun
S
Tang
State Key Laboratory of Information Engineering in Surveying Mapping and Remote Sensing, Wuhan University, 129 Luoyu Road, Wuhan 430079, China. zhuq66@263.net.|State-Province Joint Engineering Laboratory of Spatial Information Technology for High Speed Railway Safety, Chengdu 610031, China. zhuq66@263.net.|Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 610031, China. zhuq66@263.net.|Collaborative Innovation Center for Geospatial Techneology, 129 Luoyu Road, Wuhan 430079, China. zhuq66@263.net.
Qing
Q
Zhu
Department of Land Surveying & Geo-Informatics, The Hong Kong Polytechnic University, Hung Hom 999077, Hong Kong, China. wu.chen@polyu.edu.hk.
Wu
W
Chen
Department of Land Surveying & Geo-Informatics, The Hong Kong Polytechnic University, Hung Hom 999077, Hong Kong, China. w.darwish@connect.polyu.hk.
Walid
W
Darwish
Department of Land Surveying & Geo-Informatics, The Hong Kong Polytechnic University, Hung Hom 999077, Hong Kong, China. bo.wu@polyu.edu.hk.
Bo
B
Wu
Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 610031, China. huhan@whu.edu.cn.
Han
H
Hu
Faculty of Geosciences and Environmental Engineering, Southwest Jiaotong University, Chengdu 610031, China. minchen@home.swjtu.edu.cn.
Min
M
Chen
RGB-D camera; camera pose; depth; image; indoor modeling; registration
IEEE Trans Pattern Anal Mach Intell. 2004 Jun;26(6):756-77
18579936
Sensors (Basel). 2012;12(2):1437-54
22438718
Sensors (Basel). 2014 Mar 11;14(3):4981-5003
24618780
Sensors (Basel). 2015 Jun 19;15(6):14639-60
26102492