title: 3D Finger CAPE: Clicking Action and Position Estimation under Self-Occlusions in Egocentric Viewpoint.
abstract: In this paper we present a novel framework for simultaneous detection of click action and estimation of occluded fingertip positions from egocentric viewed single-depth image sequences. For the detection and estimation, a novel probabilistic inference based on knowledge priors of clicking motion and clicked position is presented. Based on the detection and estimation results, we were able to achieve a fine resolution level of a bare hand-based interaction with virtual objects in egocentric viewpoint. Our contributions include: (i) a rotation and translation invariant finger clicking action and position estimation using the combination of 2D image-based fingertip detection with 3D hand posture estimation in egocentric viewpoint. (ii) a novel spatio-temporal random forest, which performs the detection and estimation efficiently in a single framework. We also present (iii) a selection process utilizing the proposed clicking action detection and position estimation in an arm reachable AR/VR space, which does not require any additional device. Experimental results show that the proposed method delivers promising performance under frequent self-occlusions in the process of selecting objects in AR/VR space whilst wearing an egocentric-depth camera-attached HMD.
authors.lastname: Youngkyoon Jang
authors.lastname: Seung-Tak Noh
authors.lastname: Hyung Jin Chang
authors.lastname: Tae-Kyun Kim
authors.lastname: Woontack Woo
mesh_terms: D000328:Adult; D000465:Algorithms; D003196:Computer Graphics; D005385:Fingers / Q000502:physiology*; D006801:Humans; D021621:Imaging, Three-Dimensional / Q000379:methods*; D008297:Male; D014584:User-Computer Interface; D055815:Young Adult