Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion
Hand Gesture Recognition (HGR) is a form of perceptual computing that allows artificial systems to capture and interpret human gestures. HGR has applications in human-machine interaction, virtual reality, augmented reality, and human behavior analysis. The human hand can assume a near-infinite numbe...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Published: |
AUC Knowledge Fountain
2023
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| Summary: | Hand Gesture Recognition (HGR) is a form of perceptual computing that allows artificial systems to capture and interpret human gestures. HGR has applications in human-machine interaction, virtual reality, augmented reality, and human behavior analysis. The human hand can assume a near-infinite number of poses and orientations to form myriad gestures, thus increasing the difficulty of the HGR task.
The hand skeleton of connected joints effectively describes the hand’s geometric shape and thus contains richer semantic gesture information while eliminating noise from individual differences in physical hand characteristics. The efficacy and computational efficiency of skeleton-based HGR frameworks can be significantly enhanced by transforming the hand gesture recognition task into an image classification task, taking advantage of the success of ML algorithms/architectures in the image classification domain.
This research thesis document presents a skeleton-based HGR framework that implements temporal information condensation via data-level fusion to encode spatiotemporal gesture information into RGB images. The spatiotemporal images are processed through an end-to-end ensemble multi-stream CNN architecture that generates the class probabilities.
The framework was extensively evaluated on the CNR, FPHA, LMDHG, SHREC2017, and DHG1428 HGR benchmark datasets. The framework’s performance was competitive with the SOTA for the evaluation datasets, within -4.10% and +6.86% of reported classification accuracies. The experimental results confidently demonstrated the viability of the temporal information condensation and data-level fusion techniques within the HGR domain. |
|---|