Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Dynamic Hand Gesture Recognition Using Ultrasonic Sonar Sensors and Deep Learning
The space of hand gesture recognition using radar and sonar is dominated mostly by radar applications. In addition, the machine learning algorithms used by these systems are typically based on convolutional neural networks with some applications exploring the use of long short term memory networks....
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
Department of Electrical Engineering
2022
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613140973256704 |
|---|---|
| access_status_str | Open Access |
| author | Lin, Chiao-Shing |
| author2 | Abdul, Gaffar Mohammed Yunus |
| author_browse | Abdul, Gaffar Mohammed Yunus Lin, Chiao-Shing |
| author_facet | Abdul, Gaffar Mohammed Yunus Lin, Chiao-Shing |
| author_sort | Lin, Chiao-Shing |
| collection | Thesis |
| description | The space of hand gesture recognition using radar and sonar is dominated mostly by radar applications. In addition, the machine learning algorithms used by these systems are typically based on convolutional neural networks with some applications exploring the use of long short term memory networks. The goal of this study was to build and design a Sonar system that can classify hand gestures using a machine learning approach. Secondly, the study aims to compare convolutional neural networks to long short term memory networks as a means to classify hand gestures using sonar. A Doppler Sonar system was designed and built to be able to sense hand gestures. The Sonar system is a multi-static system containing one transmitter and three receivers. The sonar system can measure the Doppler frequency shifts caused by dynamic hand gestures. Since the system uses three receivers, three different Doppler frequency channels are measured. Three additional differential frequency channels are formed by computing the differences between the frequency of each of the receivers. These six channels are used as inputs to the deep learning models. Two different deep learning algorithms were used to classify the hand gestures; a Doppler biLSTM network [1] and a CNN [2]. Six basic hand gestures, two in each x- y- and z-axis, and two rotational hand gestures are recorded using both left and right hand at different distances. The gestures were also recorded using both left and right hands. Ten-Fold cross-validation is used to evaluate the networks' performance and classification accuracy. The LSTM was able to classify the six basic gestures with an accuracy of at least 96% but with the addition of the two rotational gestures, the accuracy drops to 47%. This result is acceptable since the basic gestures are more commonly used gestures than rotational gestures. The CNN was able to classify all the gestures with an accuracy of at least 98%. Additionally, The LSTM network is also able to classify separate left and right-hand gestures with an accuracy of 80% and The CNN with an accuracy of 83%. The study shows that CNN is the most widely used algorithm for hand gesture recognition as it can consistently classify gestures with various degrees of complexity. The study also shows that the LSTM network can also classify hand gestures with a high degree of accuracy. More experimentation, however, needs to be done in order to increase the complexity of recognisable gestures. |
| format | Thesis |
| id | oai:open.uct.ac.za:11427/35900 |
| institution | University of Cape Town (South Africa) |
| language | eng |
| last_indexed | 2026-06-10T12:31:24.573Z |
| license_str | Not specified — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from UCTD — University of Cape Town Open Access Repository |
| publishDate | 2022 |
| publishDateRange | 2022 |
| publishDateSort | 2022 |
| publisher | Department of Electrical Engineering |
| publisherStr | Department of Electrical Engineering |
| record_format | dspace |
| source_str | UCTD — University of Cape Town Open Access Repository |
| spelling | oai:open.uct.ac.za:11427/35900 Dynamic Hand Gesture Recognition Using Ultrasonic Sonar Sensors and Deep Learning Lin, Chiao-Shing Abdul, Gaffar Mohammed Yunus Son, Jarryd Sonar hand gesture recognition LSTM CNN human-computer interaction The space of hand gesture recognition using radar and sonar is dominated mostly by radar applications. In addition, the machine learning algorithms used by these systems are typically based on convolutional neural networks with some applications exploring the use of long short term memory networks. The goal of this study was to build and design a Sonar system that can classify hand gestures using a machine learning approach. Secondly, the study aims to compare convolutional neural networks to long short term memory networks as a means to classify hand gestures using sonar. A Doppler Sonar system was designed and built to be able to sense hand gestures. The Sonar system is a multi-static system containing one transmitter and three receivers. The sonar system can measure the Doppler frequency shifts caused by dynamic hand gestures. Since the system uses three receivers, three different Doppler frequency channels are measured. Three additional differential frequency channels are formed by computing the differences between the frequency of each of the receivers. These six channels are used as inputs to the deep learning models. Two different deep learning algorithms were used to classify the hand gestures; a Doppler biLSTM network [1] and a CNN [2]. Six basic hand gestures, two in each x- y- and z-axis, and two rotational hand gestures are recorded using both left and right hand at different distances. The gestures were also recorded using both left and right hands. Ten-Fold cross-validation is used to evaluate the networks' performance and classification accuracy. The LSTM was able to classify the six basic gestures with an accuracy of at least 96% but with the addition of the two rotational gestures, the accuracy drops to 47%. This result is acceptable since the basic gestures are more commonly used gestures than rotational gestures. The CNN was able to classify all the gestures with an accuracy of at least 98%. Additionally, The LSTM network is also able to classify separate left and right-hand gestures with an accuracy of 80% and The CNN with an accuracy of 83%. The study shows that CNN is the most widely used algorithm for hand gesture recognition as it can consistently classify gestures with various degrees of complexity. The study also shows that the LSTM network can also classify hand gestures with a high degree of accuracy. More experimentation, however, needs to be done in order to increase the complexity of recognisable gestures. 2022-03-04T07:52:48Z 2022-03-04T07:52:48Z 2021 2022-03-03T12:38:39Z Master Thesis Masters MSc http://hdl.handle.net/11427/35900 eng application/pdf Department of Electrical Engineering Faculty of Engineering and the Built Environment |
| spellingShingle | Sonar hand gesture recognition LSTM CNN human-computer interaction Lin, Chiao-Shing Dynamic Hand Gesture Recognition Using Ultrasonic Sonar Sensors and Deep Learning |
| thesis_degree_str | Master's |
| title | Dynamic Hand Gesture Recognition Using Ultrasonic Sonar Sensors and Deep Learning |
| title_full | Dynamic Hand Gesture Recognition Using Ultrasonic Sonar Sensors and Deep Learning |
| title_fullStr | Dynamic Hand Gesture Recognition Using Ultrasonic Sonar Sensors and Deep Learning |
| title_full_unstemmed | Dynamic Hand Gesture Recognition Using Ultrasonic Sonar Sensors and Deep Learning |
| title_short | Dynamic Hand Gesture Recognition Using Ultrasonic Sonar Sensors and Deep Learning |
| title_sort | dynamic hand gesture recognition using ultrasonic sonar sensors and deep learning |
| topic | Sonar hand gesture recognition LSTM CNN human-computer interaction |
| url | http://hdl.handle.net/11427/35900 |
| work_keys_str_mv | AT linchiaoshing dynamichandgesturerecognitionusingultrasonicsonarsensorsanddeeplearning |