Full Text Available
Note: Clicking the button above will open the full text document at the original institutional repository in a new window.
Real-time 3D biomechanical analysis and AI personal trainer for strength and conditioning exercises using markerless motion capture
Steyn, L. 2025. Real-time 3D Biomechanical Analysis and AI Personal Trainer for Strength and Conditioning Exercises Using Markerless Motion Capture. Unpublished masters thesis. Stellenbosch: Stellenbosch University [online]. Available: https://scholar.sun.ac.za/items/9869923a-a8bb-4eaa-a304-c38e39c7...
Saved in:
| Main Author: | |
|---|---|
| Other Authors: | |
| Format: | Thesis |
| Language: | English |
| Published: |
Stellenbosch : Stellenbosch University
2025
|
| Subjects: | |
| Tags: |
No Tags, Be the first to tag this record!
|
| _version_ | 1867613970457690112 |
|---|---|
| access_status_str | Open Access |
| author | Steyn, Lize |
| author2 | Theart, R. P. |
| author_browse | Steyn, Lize Theart, R. P. |
| author_facet | Theart, R. P. Steyn, Lize |
| author_sort | Steyn, Lize |
| collection | Thesis |
| dc_rights_str_mv | Stellenbosch University |
| description | Steyn, L. 2025. Real-time 3D Biomechanical Analysis and AI Personal Trainer for Strength and Conditioning Exercises Using Markerless Motion Capture. Unpublished masters thesis. Stellenbosch: Stellenbosch University [online]. Available: https://scholar.sun.ac.za/items/9869923a-a8bb-4eaa-a304-c38e39c71061 |
| format | Thesis |
| id | oai:scholar.sun.ac.za:10019.1/132416 |
| institution | Stellenbosch University (South Africa) |
| language | English |
| last_indexed | 2026-06-10T12:44:36.436Z |
| license_str | Other — see source repository |
| provenance_str_mv | Harvested via OAI-PMH from SUNScholar — Stellenbosch University Repository |
| publishDate | 2025 |
| publishDateRange | 2025 |
| publishDateSort | 2025 |
| publisher | Stellenbosch : Stellenbosch University |
| publisherStr | Stellenbosch : Stellenbosch University |
| record_format | dspace |
| source_str | SUNScholar — Stellenbosch University Repository |
| spelling | oai:scholar.sun.ac.za:10019.1/132416 Real-time 3D biomechanical analysis and AI personal trainer for strength and conditioning exercises using markerless motion capture Steyn, Lize Theart, R. P. Stellenbosch University. Faculty of Engineering. Dept. of Electrical and Electronic Engineering. Biomechanics -- Data processing Real-time data processing Kalman filtering Physical fitness -- Physiological aspects Fitness industry -- Technological innovations UCTD Steyn, L. 2025. Real-time 3D Biomechanical Analysis and AI Personal Trainer for Strength and Conditioning Exercises Using Markerless Motion Capture. Unpublished masters thesis. Stellenbosch: Stellenbosch University [online]. Available: https://scholar.sun.ac.za/items/9869923a-a8bb-4eaa-a304-c38e39c71061 Thesis (MEng)--Stellenbosch University, 2025. ENGLISH ABSTRACT: Biomechanical analysis allows athletes, coaches and medical professionals to monitor the performance and safety of an individual’s movements. The majority of biomechanical analysis is currently limited to 2D video or requires expensive marker-based 3D motion capture technologies. Previous work has shown that computer vision, specifically pose estimation, is a far cheaper alternative to marker-based systems. This research aims to enhance accessibility to biomechanical analysis by developing a real-time markerless 3D motion capture system using multi-angle 2D video data. Specifically, this thesis investigates the application of 2D pose estimation models to multi-view video to construct a real-time 3D pose of an individual with the intention of providing biomechanical feedback during common gym exercises. The system was developed using the 2D video and 3D Vicon keypoint SU-EMD dataset. This dataset features video from multiple subjects performing seven common strength and conditioning exercises at different speeds. Along with the video, highly accurate marker-based 3D keypoint data is provided. The 3D data was captured using a Vicon motion capture system. The dataset is used to evaluate a selection of 2D pose estimation models for a 3D real-time system. Specifically, OpenPose, Openpifpaf, AlphaPose, DCPose and YOLOv8-pose, and its sub-models, are compared in terms of accuracy and inference time. Each model is applied to each camera view and the 2D keypoints are triangulated to the same 3D space using the intrinsic and extrinsic camera parameters, as obtained through stereo calibration. The 3D keypoints are then converted to a comparable keypoint structure. Using the Vicon keypoints as ground truth data, accuracy metrics such as mean precision, percentage of correct keypoints and mean error are determined for each model. This evaluation shows that YOLOv8m-pose, despite not being the most accurate of the models, is best suited to a 3D real-time system due to its superior speed. The model has sufficient accuracy with a 49.94 mm mean error distance at an inference time of 32.10 ms per frame. The real-time 3D motion capture system uses three easy-to-use and affordable webcams that stream multi-angle video data to a YOLOv8m-pose model, which produces 2D keypoint data. Using OpenCV’s stereo calibration process the 2D data can be triangulated. Inaccuracies in estimations between consecutive frames can cause pose data to become slightly jittery, and as such a Kalman filter is implemented to smooth out the data. Specifically, a 3D constant velocity Kalman filter is selected after an accuracy, smoothness and speed comparison of different Kalman filters is performed. The filtered data is sent to the Unity game engine via a binary file. Unity allows for efficient 3D rendering using the GPU and is thus used to create a 3D interface which features a real-time custom 3D humanoid model and biomechanical feedback. The biomechanical feedback relies on computational biomechanics. A range of biomechanical metrics are selected and applied under the supervision of a biomechanics expert. Models are developed for general performance scoring and repetition tracking. The real-time 3D system and simple biomechanical analysis are successfully implemented for the exercises in the SU-EMD dataset and generalised well to live subjects. Having proven the feasibility of the system being proven, it is hoped that this research can democratise access to biomechanical analysis systems in sports and medical contexts. Finally, in an attempt to improve the field of markerless 3D pose estimation, a new dataset is captured. The SUMediPose3D is created and is a larger and more comprehensive 2D to 3D keypoint dataset. This dataset also contains highly accurate back-projected 2D keypoints. AFRIKAANSE OPSOMMING: Biomeganiese analise stel atlete, afrigters en mediese professionele persone in staat om die kwaliteit en veiligheid van ’n individu se bewegings te monitor. Die meeste biomeganiese analise is tans beperk tot 2D-video’s of vereis duur merkergebaseerde 3D-bewegingsopname tegnologie¨e. Vorige werk het getoon dat rekenaarvisie, spesifiek posisie-skatting, ’n baie goedkoper alternatief tot merkergebaseerde stelsels is. Daarom poog hierdie navorsing om toegang tot biomeganiese analise te vergemaklik deur ’n intydse merkerlose Dbewegingsopnamestelsel te ontwikkel, wat gebruik maak van multi-aansig 2D-video data. Hierdie tesis ondersoek spesifiek die toepassing van 2D-posisie-skattingmodelle op multiaansig-video om ’n intydse 3D-posisie van ’n individu te konstrueer met die doel om biomeganiese terugvoer tydens algemene gimnasiumoefeninge te bied. Hierdie stelsel is ontwikkel met behulp van die 2D-video tot 3D Vicon-sleutelpunt SU-EMD-datastel. Die datastel bevat video van verskeie proefpersone wat sewe algemene krag- en kondisioneringsoefeninge teen verskillende snelhede uitvoer. Saam met die video word hoogs akkurate merkergebaseerde 3D-sleutelpuntdata verskaf. Hierdie 3D-data is met behulp van ’n Vicon-bewegingsopnamestelsel verkry. Hierdie datastel word gebruik om ’n seleksie van 2D-posisie-skattingmodelle te evalueer vir ’n 3D intydse stelsel. Spesifiek word OpenPose, Openpifpaf, AlphaPose, DCPose en YOLOv8-pose (en YOLOv8-pose sub-modelle) vergelyk in terme van akkuraatheid en inferensietyd. Elke model word toegepas op elke kamera-aansig en die 2D-sleutelpunte word na dieselfde 3D-ruimte getrianguleer met behulp van die intrinsieke en ekstrinsieke kameraparameters, soos verkry deur stereo kalibrasie. Die 3D-sleutelpunte word dan omgeskakel na ’n vergelykbare sleutelpuntstruktuur. Met behulp van die Vicon-sleutelpunte as grondwaarheidsdata, word akkuraatheids statistieke soos gemiddelde presisie, persentasie van korrekte sleutelpunte, gemiddelde fout, ens. vir elke model bepaal. Hierdie evaluering toon dat, hoewel YOLOv8m-pose nie die akkuraatste model is nie, dit die beste geskik is vir ’n intydse 3D-stelsel as gevolg van sy vinnige spoed. Die model het ’n voldoende akkuraatheid met ’n gemiddelde foutafstand van 49.94 mm en ’n inferensietyd van 32.10 ms per foto. Die ontwikkeling van die intydse 3D-bewegingsvangstelsel word beskryf. Hierdie stelsel gebruik drie maklik-toeganklike en bekostigbare webkameras wat multi-aansig videodata stroom, waaraan YOLOv8m-pose toegepas word om 2D-sleutelpuntdata te genereer. Met behulp van OpenCV se stereo kalibrasieproses kan die 2D-data getrianguleer word. As gevolg van onakkuraathede in skattings van raam tot raam kan posisiedata effens bewe, endaarom word ’n Kalman-filter ge¨ımplementeer om die data glad te maak. Spesifiek is ’n 3Dkonstante spoed Kalman-filter gekies na ’n akkuraatheids-, gladheids- en spoedvergelyking van verskillende Kalman-filters uitgevoer is. Die gefiltreerde data word via ’n binˆere lˆeer na Unity-speletjienenjin gestuur. Unity laat doeltreffende 3D-modelle toe met behulp van die GPU en word dus gebruik om ’n intydse 3D-koppelvlak te skep. Hierdie koppelvlak bevat ’n intydse pasgemaakte 3D-humano¨ıde model en ’n biomeganiese terugvoerkoppelvlak. Die biomeganiese terugvoer is gebaseer op beginsels van biomeganika en maak staat op berekenbare biomeganika. ’n Reeks biomeganiese statistieke word gekies en toegepas onder toesig van ’n biomeganika-deskundige. Modelle word ontwikkel vir algemene prestasietelling en repetisie-opsporing. Die intydse 3D-stelsel en eenvoudige biomeganiese analise is suksesvol ge¨ımplementeer vir die oefeninge in die SU-EMD-datastel en is goed gegeneraliseer na lewende proefpersone. Met die haalbaarheid van die stelsel wat bewys is, is die hoop dat hierdie navorsing toegang tot biomeganiese analisestelsels in sport- en mediese kontekste kan demokratiseer. Laastens, in ’n poging om die veld van merkerlose 3D-posisie-skatting te verbeter, word ’n nuwe datastel geskep. Die SUMediPose3D is ontwikkel en is ’n groter en meer omvattende 2D tot 3D sleutelpunt-datastel en bevat ook hoogs akkurate teruggeprojekteerde 2Dsleutelpunte. Masters 2025-06-06T09:21:16Z 2025-06-06T09:21:16Z 2025-03 Thesis https://scholar.sun.ac.za/handle/10019.1/132416 en Stellenbosch University xvii, 116 pages : illustrations application/pdf Stellenbosch : Stellenbosch University |
| spellingShingle | Biomechanics -- Data processing Real-time data processing Kalman filtering Physical fitness -- Physiological aspects Fitness industry -- Technological innovations UCTD Steyn, Lize Real-time 3D biomechanical analysis and AI personal trainer for strength and conditioning exercises using markerless motion capture |
| title | Real-time 3D biomechanical analysis and AI personal trainer for strength and conditioning exercises using markerless motion capture |
| title_full | Real-time 3D biomechanical analysis and AI personal trainer for strength and conditioning exercises using markerless motion capture |
| title_fullStr | Real-time 3D biomechanical analysis and AI personal trainer for strength and conditioning exercises using markerless motion capture |
| title_full_unstemmed | Real-time 3D biomechanical analysis and AI personal trainer for strength and conditioning exercises using markerless motion capture |
| title_short | Real-time 3D biomechanical analysis and AI personal trainer for strength and conditioning exercises using markerless motion capture |
| title_sort | real time 3d biomechanical analysis and ai personal trainer for strength and conditioning exercises using markerless motion capture |
| topic | Biomechanics -- Data processing Real-time data processing Kalman filtering Physical fitness -- Physiological aspects Fitness industry -- Technological innovations UCTD |
| url | https://scholar.sun.ac.za/handle/10019.1/132416 |
| work_keys_str_mv | AT steynlize realtime3dbiomechanicalanalysisandaipersonaltrainerforstrengthandconditioningexercisesusingmarkerlessmotioncapture |