Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion
Hand Gesture Recognition (HGR) is a form of perceptual computing that allows artificial systems to capture and interpret human gestures. HGR has applications in human-machine interaction, virtual reality, augmented reality, and human behavior analysis. The human hand can assume a near-infinite numbe...
Saved in:
| Main Author: | |
|---|---|
| Format: | Thesis |
| Published: |
AUC Knowledge Fountain
2023
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613421946535936 |
|---|---|
| access_status_str | Open Access |
| author | Yusuf, Oluwaleke |
| author_browse | Yusuf, Oluwaleke |
| author_facet | Yusuf, Oluwaleke |
| author_sort | Yusuf, Oluwaleke |
| collection | Thesis |
| description | Hand Gesture Recognition (HGR) is a form of perceptual computing that allows artificial systems to capture and interpret human gestures. HGR has applications in human-machine interaction, virtual reality, augmented reality, and human behavior analysis. The human hand can assume a near-infinite number of poses and orientations to form myriad gestures, thus increasing the difficulty of the HGR task.
The hand skeleton of connected joints effectively describes the hand’s geometric shape and thus contains richer semantic gesture information while eliminating noise from individual differences in physical hand characteristics. The efficacy and computational efficiency of skeleton-based HGR frameworks can be significantly enhanced by transforming the hand gesture recognition task into an image classification task, taking advantage of the success of ML algorithms/architectures in the image classification domain.
This research thesis document presents a skeleton-based HGR framework that implements temporal information condensation via data-level fusion to encode spatiotemporal gesture information into RGB images. The spatiotemporal images are processed through an end-to-end ensemble multi-stream CNN architecture that generates the class probabilities.
The framework was extensively evaluated on the CNR, FPHA, LMDHG, SHREC2017, and DHG1428 HGR benchmark datasets. The framework’s performance was competitive with the SOTA for the evaluation datasets, within -4.10% and +6.86% of reported classification accuracies. The experimental results confidently demonstrated the viability of the temporal information condensation and data-level fusion techniques within the HGR domain. |
| format | Thesis |
| id | oai:fount.aucegypt.edu:etds-3017 |
| institution | American University in Cairo (Egypt) |
| last_indexed | 2026-06-10T12:35:53.165Z |
| license_str | Not specified — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from AUC Knowledge Fountain — bepress |
| publishDate | 2023 |
| publishDateRange | 2023 |
| publishDateSort | 2023 |
| publisher | AUC Knowledge Fountain |
| publisherStr | AUC Knowledge Fountain |
| record_format | dspace |
| source_str | AUC Knowledge Fountain — bepress |
| spelling | oai:fount.aucegypt.edu:etds-3017 Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion Yusuf, Oluwaleke Hand Gesture Recognition (HGR) is a form of perceptual computing that allows artificial systems to capture and interpret human gestures. HGR has applications in human-machine interaction, virtual reality, augmented reality, and human behavior analysis. The human hand can assume a near-infinite number of poses and orientations to form myriad gestures, thus increasing the difficulty of the HGR task. The hand skeleton of connected joints effectively describes the hand’s geometric shape and thus contains richer semantic gesture information while eliminating noise from individual differences in physical hand characteristics. The efficacy and computational efficiency of skeleton-based HGR frameworks can be significantly enhanced by transforming the hand gesture recognition task into an image classification task, taking advantage of the success of ML algorithms/architectures in the image classification domain. This research thesis document presents a skeleton-based HGR framework that implements temporal information condensation via data-level fusion to encode spatiotemporal gesture information into RGB images. The spatiotemporal images are processed through an end-to-end ensemble multi-stream CNN architecture that generates the class probabilities. The framework was extensively evaluated on the CNR, FPHA, LMDHG, SHREC2017, and DHG1428 HGR benchmark datasets. The framework’s performance was competitive with the SOTA for the evaluation datasets, within -4.10% and +6.86% of reported classification accuracies. The experimental results confidently demonstrated the viability of the temporal information condensation and data-level fusion techniques within the HGR domain. 2023-01-31T08:00:00Z thesis application/pdf https://fount.aucegypt.edu/etds/1986 https://fount.aucegypt.edu/context/etds/article/3017/viewcontent/Oluwaleke_Umar_Yusuf__Official_Thesis_Document.pdf Theses and Dissertations AUC Knowledge Fountain Hand Gesture Recognition Temporal Information Representation Data-Level Fusion Multi-Stream Convolutional Neural Network Transfer Learning Skeleton Ensemble Learning Artificial Intelligence and Robotics Data Science Graphics and Human Computer Interfaces Robotics |
| spellingShingle | Hand Gesture Recognition Temporal Information Representation Data-Level Fusion Multi-Stream Convolutional Neural Network Transfer Learning Skeleton Ensemble Learning Artificial Intelligence and Robotics Data Science Graphics and Human Computer Interfaces Robotics Yusuf, Oluwaleke Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion |
| title | Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion |
| title_full | Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion |
| title_fullStr | Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion |
| title_full_unstemmed | Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion |
| title_short | Skeleton-Based Hand Gesture Recognition Using Data-Level Fusion |
| title_sort | skeleton based hand gesture recognition using data level fusion |
| topic | Hand Gesture Recognition Temporal Information Representation Data-Level Fusion Multi-Stream Convolutional Neural Network Transfer Learning Skeleton Ensemble Learning Artificial Intelligence and Robotics Data Science Graphics and Human Computer Interfaces Robotics |
| url | https://fount.aucegypt.edu/etds/1986 https://fount.aucegypt.edu/context/etds/article/3017/viewcontent/Oluwaleke_Umar_Yusuf__Official_Thesis_Document.pdf |
| work_keys_str_mv | AT yusufoluwaleke skeletonbasedhandgesturerecognitionusingdatalevelfusion |