Extracting hand articulations from monocular depth images using curvature scale space descriptors

Shao-fan WANG,Chun LI,De-hui KONG,Bao-cai YIN

Front. Inform. Technol. Electron. Eng. 2016, 17 (1): 41-54. DOI: 10.1631/FITEE.1500126

Abstract

HTML

PDF (2840KB)

We propose a framework of hand articulation detection from a monocular depth image using curvature scale space (CSS) descriptors. We extract the hand contour from an input depth image, and obtain the fingertips and finger-valleys of the contour using the local extrema of a modified CSS map of the contour. Then we recover the undetected fingertips according to the local change of depths of points in the interior of the contour. Compared with traditional appearance-based approaches using either angle detectors or convex hull detectors, the modified CSS descriptor extracts the fingertips and finger-valleys more precisely since it is more robust to noisy or corrupted data; moreover, the local extrema of depths recover the fingertips of bending fingers well while traditional appearance-based approaches hardly work without matching models of hands. Experimental results show that our method captures the hand articulations more precisely compared with three state-of-the-art appearance-based approaches.

Finally, we recover the fingertip of the thumb if the thumb is bending. If all the fingers are straight except the thumb (see the third row of Fig. 10), we extract the points whose depths are smaller than the depth of the palm center, and take the farthest point from the finger-root of the thumb as the fingertip of the thumb. If the hand contains bending fingers other than the thumb (see the fourth to seventh rows of Fig. 10), we shall handle such a complicated case carefully. We collect all the points within the polygon whose vertices are five finger-roots and the first and the last point of the contour. Among this collection, we extract the points whose depth is smaller than the depth of the palm center. To separate the points of bending thumb from the points of bending non-thumb fingers, we remove the points whose depth equals the depth of any bending non-thumb fingertip and the points whose depth equals the depth of any bending non-thumb finger-root (We note that this removal involves all fingertips and finger-roots of non-thumb fingers which are bending). Finally, the finger-root of the thumb is given by the farthest point from the finger-root of the thumb among this collection. We illustrate this procedure of recovering the fingertip of the bending thumb in Fig. 4. The whole procedure for both cases is given in Algorithm 1.