Feature representation of RGB-D images using joint spatial-depth feature pooling
Research output: Contribution to journal › Journal article › Research › peer-review
Standard
Feature representation of RGB-D images using joint spatial-depth feature pooling. / Pan, Hong; Olsen, Søren Ingvor; Zhu, Yaping.
In: Pattern Recognition Letters, Vol. 80, No. 1, 2016, p. 239-248.Research output: Contribution to journal › Journal article › Research › peer-review
Harvard
APA
Vancouver
Author
Bibtex
}
RIS
TY - JOUR
T1 - Feature representation of RGB-D images using joint spatial-depth feature pooling
AU - Pan, Hong
AU - Olsen, Søren Ingvor
AU - Zhu, Yaping
PY - 2016
Y1 - 2016
N2 - Recent development in depth imaging technology makes acquisition of depth information easier. With the additional depth cue, RGB-D cameras can provide effective support for many RGB-D perception tasks beyond traditional RGB information. However, current feature representation based on RGB-D images utilizes depth information only to extract local features, without considering it to improve robustness and discriminability of the feature representation by merging depth cues into feature pooling. Spatial pyramid model (SPM) has become the standard protocol to split a 2D image plane into sub-regions for feature pooling of RGB-D images. We argue that SPM may not be the optimal pooling scheme for RGB-D images, as it only pools features spatially and completely discards their depth topological structures. Instead, we propose a novel joint spatial-depth pooling (JSDP) scheme which further partitions SPM using the depth cue and pools features simultaneously in 2D image plane and along the depth direction. By combining the JSDP with standard feature extraction and feature encoding modules, we outperform state-of-the-art methods on benchmarks for RGB-D object classification, detection and scene recognition.
AB - Recent development in depth imaging technology makes acquisition of depth information easier. With the additional depth cue, RGB-D cameras can provide effective support for many RGB-D perception tasks beyond traditional RGB information. However, current feature representation based on RGB-D images utilizes depth information only to extract local features, without considering it to improve robustness and discriminability of the feature representation by merging depth cues into feature pooling. Spatial pyramid model (SPM) has become the standard protocol to split a 2D image plane into sub-regions for feature pooling of RGB-D images. We argue that SPM may not be the optimal pooling scheme for RGB-D images, as it only pools features spatially and completely discards their depth topological structures. Instead, we propose a novel joint spatial-depth pooling (JSDP) scheme which further partitions SPM using the depth cue and pools features simultaneously in 2D image plane and along the depth direction. By combining the JSDP with standard feature extraction and feature encoding modules, we outperform state-of-the-art methods on benchmarks for RGB-D object classification, detection and scene recognition.
KW - Faculty of Science
KW - Computer Science
KW - Image analysis
KW - Computer Vision
KW - RGB-D feature representation
U2 - 10.1016/j.patrec.2016.04.001
DO - 10.1016/j.patrec.2016.04.001
M3 - Journal article
VL - 80
SP - 239
EP - 248
JO - Pattern Recognition Letters
JF - Pattern Recognition Letters
SN - 0167-8655
IS - 1
ER -
ID: 165352582